Photonic integrated circuit for optical computing and neural network

Abstract

To provide a photonic integrated circuit for optical computing and neural networks.SOLUTION: Aspects of the disclosure relate to a method of analyzing an image using an optical neural network (ONN). As an example, a method may include transmitting, by an input photonic integrated circuit (PIC), a signal including a test pattern. The test pattern may represent an image to be analyzed. The method may also include filtering, by one or more convolution PIC layers, the signal including the test pattern. The filtering may be based on a target pattern. The method may also include comparing, by an analyzer PIC, the filtered signal to a threshold.

Description

Technical Field

[0001] Cross - Reference to Related Applications

[0001] This application claims the benefit of the filing date of U.S. Provisional Patent Application No. 63 / 613,958, filed on December 22, 2023, the entire disclosure of which is incorporated herein by reference.

Background Art

[0002] Background

[0002] Image processing, recognition, and classification are one of the processes that require the most computer resources, with over 750 billion images existing on the Internet and over 3 billion images being added daily. For efficient search, these are processed using neural networks to identify key features such as color, texture, edges, patterns, etc. These features are fed into a system including a deep - learning neural network trained to understand images, and then stored in and indexed by a database. In this regard, when a customer performs a search with an input image, these systems provide the most visually similar images. Considering the large number of images, a large computational resource overhead may be required for processing. Along with the complexity of data manipulation techniques and the rapid increase in the size of datasets, highly integrated and scalable photonic hardware that is ultra - small in size and energy - consuming may be required to avoid resource bottlenecks in artificial neural networks.

Summary of the Invention

Means for Solving the Problems

[0003] Summary

[0003] Aspects of the present disclosure are directed to a method of analyzing an image using an optical neural network (ONN). The method includes transmitting, by an input photonic integrated circuit (PIC), a signal including a test pattern, wherein the test pattern represents the image to be analyzed, filtering, by one or more convolutional PIC layers, the signal including the test pattern, wherein the filtering is based on a target pattern, comparing, by an analyzer PIC, the filtered signal with a threshold, and outputting, by the analyzer PIC, a result of the comparison.

[0004]

[0004] In one example, the method further includes performing a Fourier transform (FT) on the signal including the test pattern from the input PIC in a first lens, and performing an inverse Fourier transform (IFT) on the filtered signal from one or more convolutional PIC layers in a second lens.

[0005]

[0005] In another example, outputting, by the analyzer PIC, the result of the comparison includes storing the result in the memory of the ONN.

[0006]

[0006] In a further example, the method further includes receiving, at the input PIC, the test pattern, and encoding, at the input PIC, the test pattern onto the signal. Additionally, encoding, at the input PIC, the test pattern onto the signal may include encoding components of the test pattern of the signal.

[0007]

[0007] In an additional example, the method further includes receiving a signal including a test pattern in one or more convolutional PIC layers and receiving a signal including a target pattern from a target PIC in one or more convolutional PIC layers. In addition, receiving a signal including a target pattern in one or more convolutional PIC layers may include receiving a signal including a number of components of the target pattern, the number of components of the target pattern including a first component and a second component, and filtering a signal including a test pattern by one or more convolutional PIC layers may include filtering a signal including a test pattern based on a first component in a first layer of the one or more convolutional PIC layers and filtering a signal including a test pattern based on a second component in a second layer of the one or more convolutional PIC layers. In addition, filtering a signal including a test pattern based on a first component in a first layer of the one or more convolutional PIC layers and filtering a signal including a test pattern based on a second component in a second layer of the one or more convolutional PIC layers can be performed in parallel. In addition or alternatively, filtering a signal including a test pattern based on a first component in a first layer of the one or more convolutional PIC layers and filtering a signal including a test pattern based on a second component in a second layer of the one or more convolutional PIC layers can be performed sequentially.

[0008]

[0008] In a further example, the threshold values are a number of threshold values, and each of the number of threshold values corresponds to a different component of the test pattern.

[0009]

[0009]

[0010]

[0010] In another example, the threshold value is a confidence threshold value.

[0011]

[0011] In an additional example, the result of the comparison includes determining a class of the test pattern or recognizing an image of the test pattern.

[0012]

[0012] Another aspect of the present disclosure is directed to an optical neural network (ONN) formed as a number of layers. The ONN includes a first layer including an input photonic integrated circuit (PIC), where the input PIC is configured to transmit a signal including one or more test patterns, each test pattern representing an image to be analyzed; a second layer including a first lens, where the first lens is configured to perform a Fourier transform (FT) on the signal passing therethrough; a third layer including one or more convolutional PIC layers, where the one or more convolutional PIC layers are configured to filter a received signal including one or more test patterns based on one or more target patterns; a fourth layer including a second lens, where the second lens is configured to perform an inverse Fourier transform (IFT) on the signal passing therethrough; and a fifth layer including an analyzer PIC, where the analyzer PIC is configured to compare the filtered signal from the third layer with one or more thresholds.

[0013]

[0013] In one example, the FT is a fast Fourier transform (FFT), and the IFT is an inverse fast Fourier transform (IFFT).

[0014]

[0014] In another example, the first lens and the second lens are metalenses.

[0015]

[0015] In a further example, the first lens and the second lens are stacked interlayer multiplexed planar diffraction cell layers.

[0016]

[0016] In an additional example, as a result of the comparison, a determination of the class of the test pattern or recognition of the image of the test pattern is performed.

[0017]

[0017] Another aspect of the present disclosure is directed to an optical neural network (ONN) formed as a number of layers. The ONN includes a first layer including an input photonic integrated circuit (PIC), where the input PIC is configured to transmit a signal beam including one or more test patterns, each test pattern representing an image to be analyzed; a second layer including one or more convolutional PIC layers, where the one or more convolutional PIC layers are configured to filter a received signal including one or more test patterns based on one or more target patterns; and a third layer including an analyzer PIC, where the analyzer PIC is configured to compare the filtered optical beam from the second layer with a threshold.

[0018]

[0018] In one example, a first lens configured to perform a Fourier transform (FT) on the optical beam passing therethrough is included in one of i) the first layer, ii) the second layer, iii) the third layer.

[0019]

[0019] In an additional example, a second lens configured to perform an inverse Fourier transform (IFT) on the optical beam passing therethrough is included in one of i) the first layer, ii) the second layer, iii) the third layer.

Brief Description of the Drawings

[0020] Brief Description of the Drawings

Figure 1

[0020] A block diagram according to an aspect of the present disclosure.

Figure 2

[0021] A drawing diagram according to an aspect of the present disclosure.

Figure 3A

[0022] A block diagram according to an aspect of the present disclosure.

Figure 3B

[0023] A drawing diagram according to an aspect of the present disclosure.

Figure 4A

[0024] A drawing diagram according to an aspect of the present disclosure.

Figure 4B

[0025] A drawing diagram according to an aspect of the present disclosure.

Figure 5

[0026] A flowchart according to an aspect of the present disclosure.

Mode for Carrying Out the Invention

[0021] Detailed Description Overview

[0027] The present technology relates to an optical neural network (ONN) implemented using a photonic integrated circuit (PIC). The ONN implemented using the PIC can be used in image analysis. The ONN can include a number of layers. The number of layers can include an input PIC, a first lens, one or more convolutional PIC layers, a second lens, and an analyzer PIC. In some examples, a target PIC can be included in one or more convolutional layers. In this regard, the number of layers of the ONN can be used in image analysis. The number of layers can analyze an image through processing when the image is encoded on a signal. The signal can be an optical-based signal, including in some cases an optical beam including a communication signal. The analysis of the image can serve different purposes. Thus, in one example, the ONN can classify an image. In another example, the ONN can recognize an image. In addition, one or more of the number of layers can include an optical phased array (OPA). The OPA can be one-dimensional or two-dimensional (2D).

[0022]

[0028] Generally, an ONN implemented using a free-space approach may require a complex structure and components lacking in scalability. In this regard, its components require a large-scale auxiliary device group lacking in the scalability required for a useful ONN.

[0023]

[0029] To address this, as described above, an ONN can be implemented using a PIC-based OPA. In this regard, the ONN can utilize the scalability of the OPA to perform operations such as multiplying complex-valued matrices and vectors, and has advantages such as ultra-high bandwidth, high computational speed, and high parallelism compared to electronic counterparts. The PIC approach composed of OPAs takes advantage of the progress of complementary metal-oxide-semiconductor (CMOS) manufacturing technology to replace bulky centimeter-sized ONN components with integrated micron-level semiconductor photonic constructs. These CMOS-compatible photonics can be directly integrated with CMOS electronics and provide a complete ONN chip such as an electronic central processing unit (CPU) or a graphics processing unit (GPU). Also, this enables the manufacture of ONN chips at the volume and cost scales required for next-generation wide-ranging speech recognition, image classification, computer vision, and natural language processing applications.

[0024]

[0030] In addition, the extra dimensions enable system improvements such as wavelength division, polarization, and spatial mode multiplexing, provide multi-threaded processing with little extra computational overhead, and result in extremely low energy consumption that can drive the system's key performance indicators.

[0025]

[0031] As discussed above, the ONN implemented using PIC can be used in a method for analyzing an image. The analysis of the image can be targeted at different purposes. In one example, the ONN can classify an image (e.g., perform an image classification task). In such an example, a test pattern can be analyzed to determine the class of the image to which it belongs. In other examples, the ONN can recognize a specific pattern or other information and / or the image itself in the image (e.g., perform an image recognition task). In such examples, the analysis result can include whether the target pattern is included in the test pattern.

[0026]

[0032] An exemplary method of analyzing an image using ONN is to transmit a signal including a test pattern by an input photonic integrated circuit (PIC), where the test pattern can include representing the image to be analyzed. In this regard, the input PIC can be configured to receive the test pattern. The input PIC can encode the test pattern on the signal and transmit the signal. In some examples, the signal can be converted to a Fourier transform region via a first lens and received by one or more convolutional PIC layers.

[0027]

[0033] The method is to filter a signal including a test pattern by one or more convolutional PIC layers, where the filtering can further include being based on a target pattern. One or more convolutional PIC layers can apply a filter or mask based on one or more target patterns encoded on the signal. The one or more target patterns can include exemplary images to be compared with or correlated with the test pattern. In some examples, the one or more target patterns can be in the FT region. In some examples, the one or more target patterns can include training data. In some examples, the one or more target patterns can be encoded by a target PIC and received from the target PIC. The filtered signal can be transmitted by one or more convolutional PIC layers through a second lens. The filtered signal can be converted from the FT region and received by an analyzer PIC.

[0028]

[0034] The method may further include comparing the filtered signal with a threshold by an analyzer PIC. In this regard, the analyzer PIC can be configured to compare the filtered signal with one or more thresholds to make a determination. The result of the comparison can be a correlation matrix. Filtering by one or more convolutional PIC layers 418 can be repeated until the threshold is met. In some examples, filtering by one or more convolutional PIC layers can be repeated until the threshold is met for the recognized image. In addition or alternatively, filtering can be repeated for each parameter related to the class of the image.

[0029]

[0035] The method may further include outputting the result of the comparison by an analyzer PIC. In this regard, the analyzer PIC can output the result of the comparison as an output. The result of the comparison may be in the form of a correlation matrix or may include the components of the correlation matrix.

[0030]

[0036] The features and methodologies described herein can provide a scalable ONN that can process large amounts of data and perform complex-valued operations without the limitations of current electronic counterparts. In this regard, the ONN can utilize the optical phase and amplitude control and modulation capabilities provided by the PIC to perform complex-valued matrix-vector multiplications, and can have the advantages of ultra-high bandwidth, high-speed computing, high parallelism compared to electronic counterparts, and ultra-low power consumption. The PIC elements of the ONN can employ optical absolute or relative phase measurement, control, and feedback architectures to enable the improvements provided by the complex-valued ONN architecture and coherent receiver modality. Moreover, the PIC approach composed of OPAs leverages the advancements in complementary metal-oxide-semiconductor (CMOS) manufacturing technology to replace bulk centimeter-sized ONN components with integrated micron-level semiconductor photonic constructs. These CMOS-compatible photonics can be directly integrated with CMOS electronics to provide a complete ONN chip such as an electronic central processing unit (CPU) or a graphics processing unit (GPU). Also, this enables the production of ONN chips at the volume and cost scales required for next-generation wide-ranging speech recognition, image classification, computer vision, and natural language processing applications. In addition, the extra dimensions enable system improvements such as wavelength division, polarization, and spatial mode multiplexing, provide multi-threaded processing with little extra computational overhead, and drive the system's key performance metrics with extremely low energy consumption.

[0031] Exemplary System

[0037] As described above, an ONN implemented using a PIC may include a number of layers. FIG. 1 shows an exemplary ONN 102. The exemplary ONN 102 includes one or more processors 104, a memory 106, and a number of layers 112. The one or more processors 104 can be any conventional processor such as a commercially available CPU. For example, the one or more processors 104 can be one or more complementary metal oxide semiconductor (CMOS) processors. Alternatively, the one or more processors 104 can be a dedicated device such as an application specific integrated circuit (ASIC), or another hardware-based processor such as a field programmable gate array (FPGA). In FIG. 1, the one or more processors 104 and the memory 106 are functionally shown as being within the same block, but the one or more processors 104 and the memory 106 may actually include multiple processors and memories that may or may not be stored within the same physical housing. Accordingly, references to a processor or computer are understood to include references to a collection of processors or computers or memories that may or may not operate in parallel.

[0032]

[0038] The memory 106 can store information accessible by the one or more processors 104, including data 108 and instructions 110 that can be executed by the one or more processors 104. The memory 106 can be any type of computer-readable medium such as a hard drive, memory card, ROM, RAM, DVD, or other optical disk, as well as other writable and read-only memories that can store information accessible by a processor. The system and method can include the different combinations described above, whereby different portions of the data 108 and instructions 110 are stored on different types of media.

[0033]

[0039] Data 108 can be retrieved, stored, or modified by one or more processors 104 according to instructions 110. For example, although the present system and method are not limited by a particular data structure, data 108 can be stored in a computer register, a relational database as a table having a number of different fields and records, an XML document, or a flat file. Also, data 108 can be formatted in any computer-readable format, including but not limited to binary values or Unicode. As a further example only, image data can include a bitmap that includes a grid of pixels stored according to a format that is compressed or uncompressed, reversible (e.g., BMP) or irreversible (e.g., JPEG), and bitmap or vector-based (e.g., SVG), as well as computer instructions for rendering graphics. Data 108 can include any information sufficient to identify related information, such as numbers, descriptive text, proprietary codes, references to data stored in other areas of the same memory or different memories (including other network locations), or information used by functions for computing related data.

[0034]

[0040] Instructions 110 can be any set of instructions that are executed directly (such as machine code) or indirectly (such as a script) by one or more processors 104. For example, instructions 110 can be stored as computer code on a computer-readable medium. In this regard, the terms "instructions" and "program" can be used interchangeably herein. Instructions 110 can be stored in object code format for direct processing by one or more processors 104, or in any other computer language that includes a script or collection of independent source code modules that are interpreted on demand or pre-compiled. The functions, methods, and routines of instructions 110 will be described in more detail below.

[0035]

[0041] The plurality of layers may include an input PIC, a first lens or photonic structure, one or more convolutional PIC layers, a second lens or photonic structure, and an analyzer PIC. In some examples, a target PIC can be included in one or more of the convolutional layers. In some examples, the plurality of layers can be 2D layers. In this regard, the layers can be 2D scalable, thereby enabling a path to a larger format that can handle larger image files without the need to decompose into sub-images for processing.

[0036]

[0042] FIG. 2 shows an exemplary ONN configuration including a plurality of 2D layers. The layers in FIG. 2 are an input PIC 214, a first lens or photonic structure 216, one or more convolutional PIC layers 218, a second lens or photonic structure 220, an analyzer PIC 222, and a target PIC 224. One or more of the plurality of layers can be separated by a distance f. For example, as shown, the input PIC 214 and the first lens 216 are separated by a distance f, the first lens 216 and one or more convolutional PIC layers 218 are separated by a distance f, one or more convolutional PIC layers 218 and the second lens 220 are separated by a distance f, and the second lens 220 and the analyzer PIC 222 are separated by a distance f. In such an example, a 4f ONN system architecture is depicted. The distance f can be the focal length associated with the ONN component, the effective Fourier transform length associated with the first and second lenses of the ONN, or a combination thereof.

[0037]

[0043] FIG. 3A shows an exemplary block diagram of a PIC layer 300 that includes one or more laser light sources 302, a plurality of phase shifters 304, a plurality of amplitude modulators 306, a plurality of wavelength multiplexers 308, one or more OPAs 320, and an optical control (OC) subsystem 340. In some examples, the PIC layer 300 can further include a plurality of photodiodes (PDs). The plurality of PDs can be configured to measure one or more values such as intensity and / or power.

[0038]

[0044] One or more OPA320s include a microlens array 322, a number of emitters 324, a number of phase shifters 326, and a number of reflectors, partial reflectors or retroreflector photonic constructs 328. The OC subsystem 340 includes one or more OC laser light sources 350, a number of OC phase and amplitude modulators 352, and a number of OC PDs 356. The components of the OC subsystem 340 can be used for phase and wavefront control. In this regard, the OC subsystem 340 can be configured to correct static and dynamic relative or absolute phase errors across the PIC layer 300 via the generation of one or more control wavelengths. Also, the PIC layer 300 can include other photonic and CMOS electronic components that can support signal processing. The signal can be an optical-based signal, optionally including an optical beam that includes a communication signal. The components of the PIC layer 300 can be connected via a number of waveguides or optical fibers. One or more laser light sources 302 and / or one or more OC laser light sources 350 can have a narrow linewidth specification (e.g., less than 500 kHz) and can function as a local oscillator for a coherent detector array architecture within the PIC layer 300.

[0039]

[0045] FIG. 3B shows a corresponding exemplary graphical representation of the PIC layer 300 that includes a laser light source 302, a number of phase shifters 304, a number of amplitude modulators 306, a number of wavelength multiplexers or demultiplexers 308, an OPA320, an optical control (OC) subsystem 340, one or more processors 360, a waveguide 366, and a 1×N splitter 368. For clarity and ease of understanding, additional waveguides and other features are not fully depicted. Arrows 342 and 344 respectively represent the general directions when transmission signals and reception signals enter and exit the PIC layer 300. In some examples, the PIC layer 300 can further include a number of photodiodes (PDs). The number of PDs can be configured to measure one or more values such as intensity and / or power.

[0040]

[0046] The OPA320 includes an expression of a microlens array 322, a number of emitters 324, a number of phase shifters 326, and a reflector, partial reflector or retroreflector photonic structure 328. For clarity and ease of understanding, additional waveguides and other features are not depicted. Arrows 342 and 344 respectively represent the general directions when the transmission signal and the reception signal enter and exit the PIC layer 300.

[0041]

[0047] The microlens array 322 may include a number of convex microlenses that focus Rx light onto each of a number of emitters positioned at the focal points of the microlens array 322. The microlens array 322 can be arranged in a grid pattern where the pitch or distance between adjacent lenses is constant. In other examples, the microlens array 322 can be of a different arrangement, having a different number of rows and columns, different shapes and / or different pitches (consistent or inconsistent) for different lenses.

[0042]

[0048] Each microlens of the microlens array can have a diameter and / or height of several micrometers, dozens of micrometers, or thousands of micrometers. In addition, each microlens of the microlens array 322 can be manufactured by directly forming, printing, or etching the lens onto the wafer of the OPA320 of the PIC layer 300. Alternatively, the microlens array 322 can be formed, printed, or etched as a separately manufactured microlens array. In this example, the microlens array 322 can be a rectangular or square plate made of glass or silica, with a length and width of several millimeters (e.g., 20 mm or more or less) and a thickness of 0.76 mm or more or less. By incorporating the microlens array within the OPA320, it may be possible to reduce the size of the diffraction grating emitter and increase the spacing between emitters. Thus, the two-dimensional waveguide routing within the OPA may better conform to a single-layer optical phased array. In other examples, rather than a physical microlens array, an array of diffractive optical elements (DOEs) can be used to reproduce the function of the microlens array.

[0043]

[0049] Each microlens of the microlens array 322 can be associated with each emitter of a number of emitters 324. For example, each microlens can have an emitter that is the source of the Tx signal and the focus of the Rx signal. In this regard, for a given pitch (i.e., the edge length of the microlens), the focal length of the microlens can be optimized to achieve the best transmit-receive coupling to the underlying emitter. Thus, this arrangement can increase the effective fill factor of the Rx light at each emitter while also expanding the Tx light received from each emitter at the microlens before the Tx light exits the OPA320.

[0044]

[0050] A number of emitters 324 can be configured to convert radiation from a waveguide to free space, and vice versa. Also, the emitter can generate a specific phase and intensity profile to further increase the effective fill factor of the Rx light and improve the wavefront of the Tx light. The phase and intensity profiles can be determined using inverse design or other techniques in a way that takes into account how the transmitted light changes as it propagates through and passes through the microlens array. The phase profile may differ from the flat profile of a conventional diffraction grating emitter, and the intensity profile may differ from the Gaussian intensity profile of a conventional diffraction grating emitter. However, in some implementations, the emitter can be a diffraction grating emitter with a Gaussian field profile.

[0045]

[0051] A number of phase shifters 326 can enable the detection and measurement of the Rx signal, the control of the relative phase of the Tx signal, and the modification of the Tx signal to improve the fidelity of complex number calculations from layer to layer in the PIC. Each emitter can be associated with its respective phase shifter. The Rx signal received by the number of phase shifters 326 can be provided to a receiving component including a sensor, and the Tx signal from the number of phase shifters 326 can be provided to each emitter of the number of emitters 324. The architecture of the number of phase shifters 326 can include at least one layer of phase shifters having at least one phase shifter connected to one of the number of emitters 324. In some examples, the phase shifter architecture can include multiple layers of phase shifters, and the phase shifters of the first layer can be connected in series with one or more phase shifters of the second layer. In some examples, the number of emitters 324 can be a number of optical antennas.

[0046]

[0052] A number of reflectors 328 can be configured to reflect signals transmitted from a number of emitters 324, such as one or more control wavelengths represented by arrow 318b, and return them to the number of emitters 324. As shown in FIG. 3B, one or more reflectors 328 can also be a number of reflectors, each of which can correspond to each emitter of the number of emitters 324.

[0047]

[0053] One or more processors 360 can be configured to drive the OC subsystem 340 to generate one or more control wavelengths. The one or more control wavelengths can assist in phase and wavefront control. The OC subsystem 340 of FIG. 3B includes a laser light source 350, a number of OC phase and amplitude modulators 352, a number of OC PDs 356, one or more waveguide tap couplers 314, and a photonic circulator 316. The OC subsystem 340 and its one or more control wavelengths can be used as a reference for correcting static and dynamic phase errors throughout the PIC layer 300 to control the relative or absolute phase of individual emitter paths, and can enable slightly different functionality to be realized across one or more of the many layers 112 of the ONN 102. One or more processors 360 can be configured to include an OC laser light source 350 to generate one or more control wavelengths. The one or more control wavelengths can propagate along the waveguides of the PIC layer 300, as indicated by arrows 318a and 318b.

[0048]

[0054] A number of OC phase and amplitude modulators 352 can be included in one or more layers, as shown in FIG. 3B. The number of OC phase and amplitude modulators 352 can be operably coupled to a number of OC PDs 356b. The number of OC PDs 356b can be coupled to each phase and amplitude modulator. FIG. 3B further shows that the individual PDs of the number of OC PDs 356b can be coupled to two or more phase and amplitude modulators of the number of OC phase and amplitude modulators 352 via a waveguide tap coupler such as waveguide tap coupler 314. The number of OC PDs 356 can be configured to measure one or more values such as the intensity, power, and / or relative phase of one or more control wavelengths. The measured values can be used by one or more processors 360 in the control and / or analysis of one or more control wavelengths (e.g., as feedback). In this regard, the one or more processors 360 can use the measured values in the control and / or analysis of components of the OC subsystem 340 (e.g., the number of OC phase and amplitude modulators 352).

[0049]

[0055] In some examples, different PDs can be configured to measure values in different waveguides of the phase and amplitude modulator layer to determine the relative phase. For example, the configuration can include one PD that measures the intensity / power of waveguide A, one PD that measures the intensity / power of an adjacent waveguide B, and one PD that measures the relative phase between waveguide A and waveguide B by mixing two signals with one PD. Such a configuration can provide the basic building blocks of a PD measurement system for a number of PDs. In addition, the same or a similar configuration can be repeated for adjacent waveguides (e.g., B and C, C and D, etc.). The measured values measured by three exemplary PDs for waveguide A-B can be represented as A 2 , B 2 , A 2 + B 2 + 2ABcosφ AB respectively. From these three measured values, φ of the signals propagating through waveguides A and B ABAlternatively, the relative phase can be determined. Also, the measurement of the absolute phase is possible by using a part of the control wavelength as a local oscillator and coherently interfering it with the retroreflector control wavelength that travels back and forth through a number of PDs via the PIC. For this, it may be necessary to add a number of phase and amplitude modulators to the local oscillator path and phase-lock these two with the relevant feedback electronics.

[0050]

[0056] As discussed above, a number of reflectors 328 of OPA320 can be configured to reflect signals transmitted from a number of emitters 324, such as one or more control wavelengths represented by arrow 318b, and return them to the number of emitters 324. The one or more control wavelengths that are reflected and returned can propagate through the PIC layer 300 and be guided to the OC PD 356a via the circulator 316. The measured values of the signals measured by the OC PD 356a can be processed by one or more processors 360. The measured values can be used in the control and / or analysis of the components of the PIC layer 300.

[0051]

[0057] As shown, one or more processors 360 are operably connected to a number of OC phase and amplitude modulators 352 and a number of OC PDs 356. In this regard, one or more processors 360 can be configured to drive (e.g., modify the phase and / or amplitude) a number of OC phase and amplitude modulators 352. In addition, one or more processors 360 can be configured to utilize the values measured by a number of OC PDs 356 to drive a number of OC phase and amplitude modulators 352.

[0052]

[0058] For example, the measurement values from a number of OC PDs 356 as discussed above can be used in controlling (e.g., driving) a number of OC phase and amplitude modulators 352. In some examples, one or more processors 360 can use the measurement values not only to compensate for the determined phase error (e.g., in the number of OC phase and amplitude modulators 352), but also to drive a number of phase shifters 326 of the OPA 320 to provide a specific phase to a number of emitters 324.

[0053]

[0059] One or more processors 360 can be any conventional processor such as a commercially available CPU. For example, one or more processors 360 can be one or more complementary metal oxide semiconductor (CMOS) processors. Alternatively, one or more processors 360 can be a dedicated device such as an application specific integrated circuit (ASIC), or another hardware-based processor such as a field programmable gate array (FPGA). One or more processors 360 can be operably connected to a number of phase shifters 326 of the OPA 320, the laser light source 302, a number of phase shifters 304, a number of amplitude modulators 306, the laser light source 350 of the OC subsystem 340, the OC phase and amplitude modulators 352, and the OC PDs 356.

[0054]

[0060] One or more processors 360 can be configured to induce the laser signal source 302 to generate a signal (e.g., an optical beam) propagated through the waveguide 366. The 1×N splitter 368 can be configured to split the signal from the waveguide 366 so that the signal can be routed to a number of phase shifters 304 and a number of amplitude modulators 306. One or more processors 360 can be configured to drive a number of phase shifters 304 and a number of amplitude modulators 306 to enable control of one or more characteristics of the signal propagating therethrough. In some examples, one or more processors 360 can be the same as or included in one or more processors 104 of the ONN 102. In addition or in the alternative, one or more processors 360 can include at least one processor specific to the PIC layer 300. In addition or in the alternative, one or more processors 360 of the PIC layer 300 can include at least one processor specific to the OC subsystem 340.

[0055]

[0061] A number of multiplexers 308 can be configured to receive signals from a number of phase shifters 304 and a number of amplitude modulators 306 and one or more control wavelengths from the OC subsystem 340. The number of multiplexers 308 is further configured to direct such signals and one or more control wavelengths to the OPA 320. The number of multiplexers 308 can be further configured to direct the signals received by the number of emitters 324 to one or more components configured to assist in the reception and direct one or more control wavelengths reflected back to the OC subsystem 340. In some examples, one or more multiplexers or demultiplexers 308 can route signals (e.g., the reflected control wavelength 318b and the received signal 344) to one or more of the number of OC PDs 356 for detection, electronic amplification and buffering and / or analog-to-digital conversion.

[0056]

[0062] In some examples, to maximize the bandwidth, the path length of the waveguide can be matched from the 1×N splitter 368 to the multiple emitters 324 of the OPA320. To achieve operation at a smaller bandwidth or a simpler layout, the path length can be set to approximately twice the effective refractive index × wavelength, and the total path mismatch will be inversely proportional to the reduction in bandwidth. In some examples, the physical distance between the tap coupler 314 and one or more multiplexers 308 can be a length (about 100 um) such that no relative phase error occurs between the propagation signal generated by the laser light source 302 and one or more control wavelengths generated by the laser light source 350 due to manufacturing tolerances and thermal gradients. In this regard, the dispersion between the signal and one or more control wavelengths is small enough so that the relative phase error is relatively small, or it can be compensated for any difference by applying a small offset or other phase shifter control value (offset locking) by one or more phase shifters of the PIC layer 300.

[0057]

[0063] The input PIC 214 can be configured in the same or a similar manner as the PIC layer 300 of FIGS. 3A and 3B. The input PIC 214 can include additional components that can support the digitization of test patterns and the preparation of test patterns for input. In this regard, when the PIC layer 300 is configured as the input PIC 214, one or more processors 360 can be further configured to encode a test pattern or its components onto the signal transmitted via the OPA 320. The test pattern can be an image analyzed by an ONN, such as a test image. In an example where the test pattern is a test image, one or more processors 360 can be configured to digitize the test pattern, for example, into a matrix format. Alternatively, the test pattern can be received in a digitized form, such as a matrix.

[0058]

[0064] The encoded components of the test pattern can be its different features. In one example, the different components can include different color spectra (e.g., red, green, blue of visible light). Encoding the components of the test pattern can include encoding multiple wavelengths onto the signal. For example, the first encoded wavelength includes the red light component of the test pattern, the second encoded wavelength includes the green light component of the test pattern, and the third encoded wavelength includes the blue light component of the test pattern. The components of the test pattern can be generated by a laser light source 302 via a number of phase shifters 304 and a number of amplitude modulators 306, as discussed above with respect to FIGS. 3A and 3B, processed using one or more control wavelengths of the OC subsystem 340, and encoded onto the optical signal transmitted from the OPA 320.

[0059]

[0065] In some examples, a single component can be encoded onto the signal. The input PIC 214 can be configured to transmit a signal including the test pattern via the OPA 320. In addition to simultaneously processing the color of light, generally, the ONN can process multiple forms of signals that can be decomposed into a separable basis set in parallel, for example, frequency-based signals can include audio, electronics, radio frequency, X-rays, magnetic resonance, and the like.

[0060]

[0066] One or more convolutional PIC layers 218 can be configured in the same or a similar manner as the PIC layer 300 of FIGS. 3A and 3B. The one or more convolutional PIC layers 218 can include additional components that can support signal processing and / or signal filtering (e.g., matrix multiplication of test patterns and target pattern masks). In this regard, when the PIC layer 300 is configured as one or more convolutional PIC layers 218, one or more processors 360 can be configured to receive, at the OPA 320, a signal including one or more components of a test pattern from the input PIC 214. The one or more processors 360 can be configured to apply a target pattern as a filter or mask to a signal including a test pattern from the input PIC 214. The one or more target patterns can include exemplary images that are compared to or correlated with the test pattern. In some examples, the one or more target patterns can be in the Fourier transform (FT) domain. In some examples, the one or more target patterns can include training data.

[0061]

[0067] The application of the filter or mask can be achieved by one or more convolution operations. For example, if the target pattern includes an image of the alphabet letter A, the test pattern signal can be filtered using the target pattern such that only instances or peaks of the test pattern corresponding to the alphabet letter A remain. In other words, instances of the overlap between the representation of the target pattern and the representation of the test pattern can break through the filtering. The one or more processors can be further configured to transmit a filtered signal including the test pattern by driving one or more phase shifters.

[0062]

[0068] In some examples, one or more convolutional PIC layers 218 can be a number of convolutional PIC layers. Each of the number of convolutional PIC layers can be configured to filter or apply components of a target pattern to corresponding components of a test pattern. Similar to the components of the test pattern, in one example, different components of the target pattern can include different color spectra (e.g., red, green, blue of visible light). In this regard, each convolutional PIC layer can be configured to apply a filter corresponding to a component of the target pattern to a signal including the test pattern. In addition or alternatively, the target pattern can be a number of target patterns, each of which can be applied to the test pattern or its components in different layers of the number of convolutional PIC layers.

[0063]

[0069] In such examples, the filtered signals from one or more convolutional PIC layers 218 can include a plurality of wavelengths respectively corresponding to the filtering performed in each convolutional PIC layer. In some examples, each layer can apply its respective filter in parallel.

[0064]

[0070] In some examples, the filters or masks of one or more convolutional PIC layers can be dynamically changed through the use of a number of phase shifters 304 and a number of amplitude modulators 306. Such dynamic changes enable a fast comparison of the test pattern to the target pattern.

[0065]

[0071] Following filtering of the signal including the test pattern, one or more convolutional layers 218 can be configured to transmit the filtered signal to the analyzer PIC222 via the OPA320. In some examples, one or more convolutional PIC layers 218 can include a separate transmit OPA and receive OPA instead of a bidirectional OPA such as the OPA320. In this regard, one or more convolutional PIC layers 218 can be configured to receive a signal from the input PIC214 with the receive OPA and transmit the filtered signal to the analyzer PIC222 using the transmit OPA.

[0066]

[0072] In some examples, one or more convolutional PIC layers 218 can be configured to encode the components of the target pattern onto the signal in the same way as the method discussed above with respect to encoding the test pattern in the input PIC 214. Alternatively, in some examples, the target pattern can be received as a signal from the target PIC 224. The target PIC 224 can be configured in the same or a similar manner as the PIC layer 300 of FIGS. 3A and 3B. The target PIC 224 can include additional components that can support digitization of the target pattern for use as a mask in one or more convolutional PIC layers 218. When the PIC layer 300 is configured as the target PIC 224, one or more processors 360 can be further configured to encode the target pattern or its components onto the signal transmitted via the OPA 320. In an example where the target pattern is a target image, one or more processors 360 can be configured to digitize the target pattern, for example, in a matrix format. Alternatively, the target pattern can be received in a digitized form, such as a matrix. The components of the target pattern can be generated by the laser light source 302 via a number of phase shifters 304 and a number of amplitude modulators 306, processed using one or more control wavelengths of the OC subsystem 340, and encoded onto the optical signal transmitted from the OPA 320, as discussed above with respect to FIGS. 3A and 3B. The transmitted signal can then be received by one or more convolutional PIC layers 218.

[0067]

[0073] In some examples, a single component can be encoded onto the signal. The target PIC 224 can be configured to transmit a signal including the test pattern via the OPA 320. In addition to simultaneously processing the color of light, generally, an ONN can process multiple forms of signals that can be decomposed into separable basis sets in parallel. For example, frequency-based signals can include audio, electronics, radio frequency, X-rays, magnetic resonance, and the like.

[0068]

[0074] In some examples, the target PIC224 may not include the OPA320. In this regard, the target PIC224 can be connected to one or more stacked PIC layers 218 via a number of waveguides and a number of optical fibers. In some examples, the target PIC224 may include a lens or other component that can convert an encoded signal for filtering into the Fourier domain. The lens can be configured in the same way as the first lens 216 and the second lens 220 discussed in more detail below.

[0069]

[0075] The analyzer PIC222 can be configured in the same or a similar way as the PIC layer 300 of FIGS. 3A and 3B. The analyzer PIC222 may include additional components that can support direct detection or coherent detection of the filtered signal from one or more stacked PIC layers 218. One or more processors 360 can be configured to compare the filtered signal received from one or more stacked PIC layers 218 with a threshold. The threshold can be a reliability threshold. In some examples, the threshold is a number of thresholds, and each threshold corresponds to a different component of the test pattern. In this regard, the filtered signal being equal to or exceeding the threshold may indicate that the target pattern or its component is present in the test pattern. In addition, the filtered signal being below the threshold may indicate that the target pattern or its component is not present in the test pattern. The result of the comparison may be in the form of a correlation matrix or may include the components of the correlation matrix.

[0070]

[0076] In some examples, one or more processors 360 can be further configured to output the result of comparing the filtered beam with a threshold. The output result can be stored in the memory 106 of the ONN 102 and / or an external device. In one example, the result of the comparison can include whether the test pattern includes one or more components or parameters of the target pattern (e.g., determination of the class of the test pattern). In another example, the result of the comparison can include whether the target pattern is included in the test pattern (e.g., recognition of an image of the test pattern).

[0071]

[0077] In addition, one or more processors 360 can be further configured to induce the ONN 102 to repeat the filtering process until the threshold is met and / or until each component of the test pattern is filtered by the target pattern. In some examples, one or more processors 360 can be the same as one or more processors 104 of the ONN 102 and can include a central processing unit.

[0072]

[0078] The first lens 216 and the second lens 220 can each be configured to perform a Fourier transform (FT) and an inverse Fourier transform (IFT) on the signal passing therethrough. In this regard, the filtering performed in one or more convolutional PIC layers 218 can be performed in the FT domain. In some examples, the first lens 216 and the second lens 220 can each be configured to perform a fast Fourier transform (FFT) and an inverse fast Fourier transform (IFFT) on the optical beam passing therethrough.

[0073]

[0079] In some examples, the first lens 216 and the second lens 220 can be metalenses. In one example, the first lens and the second lens each have Tx and Rx capabilities and can be formed as an OPA including silicon metalenses. Such an OPA can include the same or similar components as those discussed above with respect to FIGS. 3A and B. In such an example, the OPA can be integrated on a PIC silicon wafer carrier.

[0074]

[0080] In another example, the first lens 216 and the second lens 220 can be formed as stacked interlayer multiplexed planar diffraction cell layers. In a further example, the first lens and the second lens can each be formed as a glass surface having a semiconductor layer thereon. The semiconductor layer can be formed to be able to perform FFT and IFFT respectively. In a further example, the first and second lenses can be included in a microlens array. In this regard, the lenses of the input PIC 214, one or more convolution PIC layers 218, the analyzer PIC 222, or any combination thereof, can be configured to perform FT, IFT, etc. on the signal. The first lens 216 and the second lens 220 can be bulk optical devices, microlens arrays, diffractive optical devices, microprinted lens arrays or metalenses, or any combination thereof.

[0075] Exemplary method

[0081] As discussed above, the ONN implemented using PIC can be used in a method for analyzing an image. The analysis of the image can be targeted at different purposes. In one example, the ONN can classify the image (e.g., perform an image classification task). In such an example, the test pattern can be analyzed to determine the class of the image to which it belongs. In such an example, the analysis result can include whether the test pattern contains one or more components or parameters of the target pattern. For example, if the test image contains a flower, the ONN can be configured to determine the type of flower in the image (e.g., iris). In some examples, the ONN can be further configured to determine the subclass or species of the flower (e.g., iris setosa, iris versicolor or iris virginica).

[0076]

[0082] FIG. 4A shows a flow of a method by an exemplary drawing of an ONN 400a configured to classify images. In this regard, the input PIC can be configured to receive a test pattern 430. As discussed above, the input PIC can encode the test pattern on a signal and transmit the signal. The signal is converted to the FT region via a first lens 416 and can be received by one or more convolutional PIC layers 418. As discussed above, one or more convolutional PIC layers 418 can apply a filter or mask based on one or more target patterns 440 encoded on the signal and received from the target PIC 424. The filtered signal can be transmitted by one or more convolutional PIC layers 418 through a second lens 420. The filtered signal is converted from the FT region and can be received by an analyzer PIC 422. The analyzer PIC 422 can be configured to compare the filtered signal with one or more thresholds, as discussed above. The result of the comparison can be a correlation matrix 450a. Filtering by one or more convolutional PIC layers 418 can be repeated until the threshold is met. In addition, filtering can be repeated for each parameter related to the class of the image. Then, the analyzer PIC 422 can output the result of the comparison as an output 460a. The result of the comparison may be in the form of a correlation matrix or may include the components of the correlation matrix 450a.

[0077]

[0083] In other examples, the ONN can recognize a specific pattern or other information of an image and / or the image itself (e.g., perform an image recognition task). In such examples, the analysis result can include whether the target pattern is included in the test pattern. As an example, the ONN can recognize text. In such examples, the ONN can determine whether a specific number, alphabetic character, or other character is present in the test pattern.

[0078]

[0084] Figure 4B shows a flow of a method by an exemplary rendering of the ONN 400b configured to perform image recognition. In this regard, the input PIC can be configured to receive a test pattern 430. As discussed above, the input PIC can encode the test pattern on a signal and transmit the signal. The signal is converted to the FT region via the first lens 416 and can be received by one or more convolutional PIC layers 418. As discussed above, one or more convolutional PIC layers 418 can apply a filter or mask based on one or more target patterns 440 encoded on the signal and received from the target PIC 424. The filtered signal can be transmitted by one or more convolutional PIC layers 418 through the second lens 420. The filtered signal is converted from the FT region and can be received by the analyzer PIC 422. The analyzer PIC 422 can be configured to compare the filtered signal with one or more thresholds, as discussed above. The result of the comparison can be a correlation matrix 450b. Filtering by one or more convolutional PIC layers 418 can be repeated until a threshold is met for the recognized image. Then, the analyzer PIC 422 can output the result of the comparison as an output 460b. The result of the comparison may be in the form of a correlation matrix or may include components of the correlation matrix 450b.

[0079]

[0085] Figure 5 shows an exemplary method 500 for analyzing an image using an ONN. In block 510, the method includes transmitting, by an input PIC, a signal including a test pattern, the test pattern representing the image to be analyzed. In this regard, the input PICs 214, 414 can receive the components of the test pattern 430 and encode them on a signal (e.g., an optical beam), as discussed above. Then, the input PICs 214, 414 can transmit the optical beam including the components of the test pattern 430 to one or more convolutional PIC layers 218, 418.

[0080]

[0086] In block 520, the method further includes filtering a signal beam including a test pattern by one or more convolutional PIC layers, the filtering being based on a target pattern. One or more convolutional PIC layers 218, 418 can filter a signal including the test pattern 430 received from the input PICs 214, 418, as discussed above. In this regard, one or more processors 104, 360 can apply the target pattern 440 as a filter or mask to the signal including the test pattern 430 and transmit the filtered signal (e.g., an optical beam) to the analyzer PICs 222, 422. In some examples, multiple layers of one or more convolutional PIC layers 218, 418 can be used to filter signal components sequentially or in parallel, as discussed above. In such examples, components of the target pattern can be used to filter the signal sequentially or in parallel. In some examples, the filtering can include receiving a signal including the target pattern or its components from the target PICs 224, 424, as discussed above.

[0081]

[0087] In block 530, the method further includes comparing the filtered signal with a threshold by an analyzer PIC. As discussed above, the analyzer PICs 222, 422 can be configured to compare the filtered signal (e.g., an optical beam) received from one or more convolutional PIC layers 218, 418 with a threshold. In this regard, the comparison can associate the signal with individual components or parameters of the target pattern 440. In some examples, the threshold can be a confidence threshold. In some examples, the threshold is a plurality of thresholds, and each threshold corresponds to a different component of the test pattern 430. In this regard, the filtered signal being equal to or exceeding the threshold may indicate the presence of the target pattern 440 or its components in the test pattern 430. Additionally, the filtered signal being below the threshold may indicate the absence of the target pattern 440 or its components in the test pattern 430. In some examples, the threshold utilized during an image recognition task may be greater than the threshold utilized during an image classification task. The result of the comparison may be in the form of a correlation matrix or may include components of the correlation matrix 450.

[0082]

[0088] In block 540, the method further includes outputting the result of the comparison by an analyzer PIC. In this regard, one or more processors 104, 360 or at least one of its processors operably connected to the analyzer PICs 222, 422 can be configured to output the result of the comparison between the filtered signal and the threshold. In some examples, the result can be the result of an image classification task. In this regard, the result or output 460a can include a determination of the class of the image to which the test pattern belongs (e.g., whether one or more components or parameters of the target pattern are included in the test pattern). In some examples, the result or output 460b can be the result of an image recognition task. In this regard, the result can include recognizing a specific pattern or other information and / or the image itself of the image (e.g., whether the target pattern is included in the test pattern).

[0083]

[0089] The output result can be stored in the memory 106 of the ONN102 and / or an external device. In one example, the result of the comparison may include whether the test pattern 430 includes many components or parameters of the target pattern 440. In another example, the result of the comparison may include whether the target pattern 440 is included in the test pattern 430.

[0084]

[0090] In some examples, the method may further include performing an FT on a signal including a test pattern from the input PIC in the first lens. In addition, the method may further include performing an IFT on the filtered signal from one or more convolutional PIC layers in the second lens. In this regard, the first lenses 216, 416 can perform an FT (e.g., FFT) on the signal transmitted from the input PICs 214, 414 and received by one or more convolutional PIC layers 218, 418. In addition, the second lenses 220, 420 can perform an IFT (e.g., IFFT) on the filtered signal transmitted from one or more convolutional PIC layers 218 and received by the analyzer PICs 222, 422. Therefore, the filtering performed in one or more convolutional PIC layers 218, 418 can be performed in the FT domain.

[0085]

[0091] In some examples, the method is to filter a signal including a test pattern by one or more convolutional PIC layers, and the filtering may further include being based on a second target pattern. In this regard, one or more processors 104, 360 or at least one of the processors operably connected to the analyzer PICs 222, 422 can be configured to induce the ONN 102 to repeat the filtering process if a threshold is not met. Accordingly, the filtering process can be repeated using additional test patterns until the threshold is met. In addition or alternatively, one or more processors 104, 360 of the ONN 102 can be configured to induce the ONN 102 to repeat the filtering process until each component of the test pattern is filtered by the target pattern.

[0086]

[0092] The features and methodologies described herein can provide a scalable ONN that can process large amounts of data and perform complex number operations without electronic counterparts. In this regard, the ONN can utilize the scalability of the OPA to perform operations such as multiplying complex number matrices and vectors, and has the advantages of ultra-high bandwidth, high computing speed, high parallelism compared to electronic counterparts, and ultra-low power consumption. Moreover, the PIC approach composed of OPA utilizes the progress of complementary metal oxide semiconductor (CMOS) manufacturing technology to replace bulk centimeter-sized ONN components with integrated micron-level semiconductor photonic constructs. These CMOS-compatible photonics can be directly integrated with CMOS electronics to provide a complete ONN chip such as an electronic central processing unit (CPU) or a graphics processing unit (GPU). Also, this enables the production of ONN chips at the volume and cost scales required for next-generation wide-ranging speech recognition, image classification, computer vision, and natural language processing applications. In addition, the extra dimensions enable system improvements such as wavelength division and spatial mode multiplexing, provide multi-threaded processing with little extra computational overhead, and result in extremely low energy consumption that can drive the system's key performance indicators.

[0087]

[0093] Unless otherwise specified, the foregoing alternative examples are not mutually exclusive and can be implemented in various combinations to achieve their respective advantages. These and other variations and combinations of the features discussed above can be utilized without departing from the subject matter defined by the claims. Therefore, the foregoing description of the embodiments should be taken as illustrative rather than as a limitation of the subject matter defined by the claims. In addition, the provision of the examples described herein, phrases such as "such as," "including," and the like, and the like, should not be construed as limiting the subject matter of the claims to specific examples, but rather, these examples are only intended to illustrate one of many possible embodiments. Further, the same reference numbers in different drawings can identify the same or similar elements.

Description of Reference Numerals

[0088] 104 Processor 106 Memory 108 Data 110 Instruction 112 Layer 214 Input PIC 216 First Lens 218 Convolution PIC Layer 220 Second Lens 222 Analyzer PIC 224 Target PIC 300 PIC Layer 302 Laser Light Source 304 Phase Shifter 306 Amplitude Modulator 308 Multiplexer 320 OPA 322 Microlens Array 324 Emitter 326 Phase Shifter 328 Reflector 340 OC Subsystem 350 OC Laser Light Source 352 OC Phase and Amplitude Modulator 360 Processor 366 Waveguide 414 Input PIC 416 First lens 418 Convolution PIC layer 420 Second lens 422 Analyzer PIC 424 Target PIC 430 Test pattern 440 Target pattern 450a Correlation matrix 450b Parameter measurement matrix 460a Output 460b Output

Claims

1. A method for analyzing an image using an optical neural network (ONN), comprising: transmitting, by an input photonic integrated circuit (PIC), a signal including a test pattern, wherein the test pattern represents the image to be analyzed; filtering, by one or more convolutional PIC layers, the signal including the test pattern, wherein the filtering is based on a target pattern; comparing, by an analyzer PIC, the filtered signal with a threshold; outputting, by the analyzer PIC, a result of the comparison The method includes.

2. performing a Fourier transform (FT) on the signal including the test pattern from the input PIC in a first lens; performing an inverse Fourier transform (IFT) on the filtered signal from the one or more convolutional PIC layers in a second lens The method according to claim 1, further comprising.

3. The method according to claim 1, wherein outputting the result of the comparison by the analyzer PIC includes storing the result in the memory of the ONN.

4. receiving, in the input PIC, the test pattern; encoding, in the input PIC, the test pattern onto the signal The method according to claim 1, further comprising.

5. The method according to claim 4, wherein encoding the test pattern onto the signal in the input PIC includes encoding components of the test pattern of the signal.

6. receiving, in one or more convolutional PIC layers, the signal including the test pattern; receiving, in the one or more convolutional PIC layers, a signal including the target pattern from a target PIC The method according to claim 1, further comprising.

7. Receiving, in the one or more convolutional PIC layers, the signal including the target pattern includes receiving a signal including a large number of components of the target pattern, wherein the large number of components of the target pattern includes a first component and a second component. Filtering the signal including the test pattern by the one or more convolutional PIC layers includes filtering the signal including the test pattern based on the first component in the first layer of the one or more convolutional PIC layers, and filtering the signal including the test pattern based on the second component in the second layer of the one or more convolutional PIC layers. The method according to claim 6.

8. Filtering the signal including the test pattern based on the first component in the first layer of the one or more convolutional PIC layers and filtering the signal including the test pattern based on the second component in the second layer of the one or more convolutional PIC layers are executed in parallel. The method according to claim 7.

9. Filtering the signal including the test pattern based on the first component in the first layer of the one or more convolutional PIC layers and filtering the signal including the test pattern based on the second component in the second layer of the one or more convolutional PIC layers are executed sequentially. The method according to claim 7.

10. The method according to claim 1, wherein the threshold values are a plurality of threshold values, and each of the plurality of threshold values corresponds to a different component of the test pattern.

11. The method according to claim 1, wherein the threshold value is a confidence threshold value.

12. The method according to claim 1, wherein the result of the comparison includes determination of the class of the test pattern or recognition of an image of the test pattern.

13. An optical neural network (ONN) formed as a plurality of layers, A first layer including an input photonic integrated circuit (PIC), wherein the input PIC is configured to transmit a signal including one or more test patterns, and each test pattern represents an image to be analyzed. The first layer, A second layer including a first lens, wherein the first lens is configured to perform a Fourier transform (FT) on the signal passing therethrough. The second layer, A third layer including one or more convolutional PIC layers, wherein the one or more convolutional PIC layers are configured to filter a received signal including one or more test patterns based on one or more target patterns. A fourth layer including a second lens, wherein the second lens is configured to perform an inverse Fourier transform (IFT) on a signal passing therethrough. A fifth layer including an analyzer PIC, wherein the analyzer PIC is configured to compare the filtered signal from the third layer with one or more thresholds. Including, an ONN.

14. The ONN according to claim 13, wherein the FT is a fast Fourier transform (FFT) and the IFT is an inverse fast Fourier transform (IFFT).

15. The ONN according to claim 13, wherein the first lens and the second lens are metalenses.

16. The ONN according to claim 13, wherein the first lens and the second lens are stacked interlayer multiplexed planar diffraction cell layers.

17. The ONN according to claim 13, wherein as a result of the comparison, a determination of a class of a test pattern or recognition of an image of the test pattern is performed.

18. An optical neural network (ONN) formed as a plurality of layers, A first layer including an input photonic integrated circuit (PIC), wherein the input PIC is configured to transmit a signal beam including one or more test patterns, and each test pattern represents an image to be analyzed. A second layer including one or more convolutional PIC layers, wherein the one or more convolutional PIC layers are configured to filter a received signal including one or more test patterns based on one or more target patterns. A third layer including an analyzer PIC, wherein the analyzer PIC is configured to compare the filtered optical beam from the second layer with a threshold. Including, an ONN.

19. The ONN according to claim 18, wherein a first lens configured to perform a Fourier transform (FT) on an optical beam passing therethrough is included in one of i) the first layer, ii) the second layer, iii) the third layer.

20. A second lens configured to perform an inverse Fourier transform (IFT) on the light beam passing therethrough is included in one of i) the first layer, ii) the second layer, iii) the third layer, the ONN according to claim 18.

Images