Spatial light modulation diffraction optical neural network online learning device and imaging method

Abstract

The invention discloses a spatial light modulation diffraction light neural network on-line learning device and an imaging method. The device comprises the following steps: providing a monochromatic light beam by a laser device, and expanding the monochromatic light beam by a beam expanding lens; the HWP changes the polarization direction of the polarized light and modulates the polarization state of the light beam on the incident amplitude modulation array into p polarized light; the high-power reflector changes the light propagation direction and emits the light to the diffraction layer; the diffraction layer performs amplitude regulation and phase modulation on the light beams to obtain final output of the diffraction light neural network; the processing module collects the output light field image of each diffraction layer of the diffracted light neural network acquired by each CMOS and the current phase modulation parameter, and determines the phase modulation parameter of each diffraction layer according to a set loss function to adjust the gradient; and the control module loads the phase modulation parameters to the reflective phase type SLM of the corresponding diffraction layer. The scheme can adapt to a complex and changeable actual environment, the performance and adaptability of the diffraction light neural network are optimized, and wide application and development of the diffraction light neural network in the fields of high-performance calculation and artificial intelligence are promoted.

Description

Technical Field

[0001] The present invention relates to the field of diffraction optical neural networks, and in particular to an online learning device and an imaging method for a spatial light modulation diffraction optical neural network. Background Art

[0002] The traditional von Neumann architecture faces challenges due to the separation of storage and computing, which leads to the emergence of power consumption wall and memory wall, and the slowdown of Moore's Law further exacerbates the complexity of these problems. At the same time, the diffractive optical neural network combines the advantages of high-speed optical communication, optical interconnection, optical integration and silicon-based optoelectronic technology, and has the characteristics of ultra-high speed, large bandwidth and multi-dimensionality. In the field of high-performance computing and artificial intelligence, it shows a wide range of application prospects and has become a competitive solution to break through the limits of traditional microelectronic computing.

[0003] At present, diffraction optical neural networks usually use 3D printing or lithography technology to manufacture the diffraction layer. However, these traditional methods may limit the flexibility of their structural adjustment, resulting in the limitation of relatively single network functions. In addition, the phase modulation parameters of the neural network are usually trained on a computer based on theoretical models, which may be significantly different from the environmental variables in actual application scenarios, and may even ignore key practical influencing factors. Therefore, it is very necessary to provide a method that can use the basic principles of spatial light modulators to update and process the phase modulation parameters of each layer of the diffraction optical neural network according to the received data in actual application scenarios. Summary of the invention

[0004] In view of this, the main purpose of the present invention is to provide a spatial light modulation diffraction optical neural network online learning device and imaging method.

[0005] In order to achieve the above-mentioned object, the first aspect of the present application provides a spatial light modulation diffraction optical neural network online learning device, the device comprising:

[0006] Laser, beam expansion lens, HWP, high-power reflector, diffraction layer, processing module, control module;

[0007] The laser is used to provide a continuous and stable light source, and the light it emits is a monochromatic beam and a linearly polarized light with a fixed polarization direction;

[0008] The beam expansion lens is used to expand the monochromatic light beam, so that the light beam irradiated on the subsequent reflective SLM and CMOS is parallel light and can fill the entire modulation array;

[0009] The HWP is used to change the polarization direction of polarized light and modulate the polarization state of the light beam incident on the amplitude modulation array into p-polarized light;

[0010] The high-power reflector is used to change the light propagation direction of the light beam and emit it to the diffraction layer;

[0011] The diffraction layer is provided with several layers, which are used to obtain the final output of the diffraction optical neural network after amplitude regulation and phase modulation of the light beam;

[0012] The processing module is used to collect the output light field images of each diffraction layer of the diffraction optical neural network acquired by each CMOS and the current phase modulation parameters, and to determine the gradient to adjust the phase modulation parameters of each diffraction layer according to the set loss function;

[0013] The control module is used to load the phase modulation parameters onto the reflective phase-type SLM of the corresponding diffraction layer.

[0014] In some embodiments, each of the diffraction layers includes a PBS, a reflective amplitude-type SLM, a HWP, a CMOS, or a reflective phase-type SLM;

[0015] Two PBSs are provided, each for splitting the incident light into two perpendicular linearly polarized lights, wherein the p-polarized light completely passes through, and the s-polarized light is reflected at an angle of 45°, and the outgoing direction forms an angle of 90° with the p-polarized light;

[0016] The reflective amplitude-type SLM is used to perform amplitude modulation on incident p-polarized light;

[0017] The HWP is used to change the polarization direction of the reflected light beam after the reflective amplitude-type SLM performs amplitude modulation on the light field;

[0018] The CMOS is used to obtain the intensity of the light field after the reflective amplitude-type SLM performs amplitude modulation;

[0019] The reflective phase-type SLM is used to phase modulate the incident p-polarized light, and serves as a diffraction layer of the diffraction optical neural network.

[0020] In some embodiments, a HWP is disposed between two of the PBSs.

[0021] In some embodiments, a HWP is disposed between the light emitting side and the light incident side of two adjacent diffraction layers.

[0022] In some embodiments, the PBS uses an adhesive to glue two right-angle prisms made of negative uniaxial crystals along the inclined surface, the optical axes of the two negative uniaxial crystals are parallel to the right-angle surface through which light passes and the incident surface, the e-light is p-polarized light, and the o-light is s-polarized light.

[0023] To achieve the above-mentioned purpose, the second aspect of the present application provides an imaging method applied to the spatial light modulation diffraction optical neural network online learning device, the method comprising:

[0024] Expand and phase modulate continuous and stable monochromatic linear polarized light;

[0025] The target grayscale image is loaded on the reflective amplitude-type SLM to perform amplitude modulation on the incident p-polarized light, so that the intensity of the reflected light field is distributed according to the grayscale value of the loaded grayscale image;

[0026] The reflected light of the reflective SLM is polarized again;

[0027] After polarization modulation again, the light beam is split by the PBS, and the p component is incident on the reflective phase-type SLM along the original optical path, and the s component is reflected onto the CMOS;

[0028] The processing module calculates the loss value and gradient of the model according to the set loss function, and the control module continuously adjusts the phase modulation parameters according to the gradient until the loss value meets the set conditions and outputs the final image output.

[0029] In some embodiments, the beam expansion and phase modulation of the continuous and stable monochromatic linearly polarized light specifically includes: the laser emits a continuous and stable monochromatic linearly polarized laser beam, and the beam diameter is expanded by a beam expander lens, so that the original small-diameter beam is adjusted to a large-diameter parallel light, and the incident light wave is a plane wave, and its complex amplitude is expressed as

[0030] HWP performs polarization modulation on the expanded light beam to make it p-polarized light. The modulation process is to rotate the half-wave plate so that its fast axis direction forms an angle θ with the original polarization direction, and add a phase delay of π. After the linearly polarized light passes through the half-wave plate, it is still linearly polarized light and the orientation of the vibration plane is rotated by 2θ compared to the incident light.

[0031] In some embodiments, the target grayscale image is loaded on the reflective amplitude-type SLM to perform amplitude modulation on the incident p-polarized light so that the intensity of the reflected light field is distributed according to the grayscale value of the loaded grayscale image, specifically including: PBS performs polarization filtering on the p-polarized light beam after expansion, which can transmit p-polarized light and reflect s-polarized light, so that only p-polarized light is incident on the amplitude modulation array of the reflective amplitude-type SLM.

[0032] In some embodiments, the reflected light field of the reflective amplitude-type SLM is the input I of the first diffraction network. 1_in , Where I(x0, y0) is the grayscale value of the pixel at (x0, y0) of the written image. The light field at this time can be regarded as a linear combination of several point light sources.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present invention utilizes SLM to form an integrated system of a diffraction optical neural network. By pre-debugging and packaging all components, the trouble of repeatedly adjusting the positions of components is avoided, which greatly improves the convenience and operation efficiency of the system. The SLM is used to replace the diffraction layer of the traditional 3D printed diffraction optical neural network, and different trained phase modulation parameters can be loaded according to actual needs to achieve personalized customization for different targets, so that the network has the characteristics of adjustable parameters and reusability. In addition, the present invention utilizes a processing module to calculate the gradient and phase modulation parameters of the multi-layer diffraction network in real time in a specific physical model. When using it, it is possible to not consider the influence of environmental factors in advance, but only continuously calculate and adjust the phase modulation parameters of the phase-type SLM according to the results, thereby changing the modulation effect of the diffraction optical neural network, and has the characteristics of high real-time performance and strong anti-interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings described herein are used to disclose a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0036] Figure 1 A schematic diagram of an online learning device for a spatial light modulated diffraction optical neural network in one embodiment;

[0037] Figure 2 It is a flow chart of an imaging method of a spatial light modulation diffraction optical neural network online learning device in one embodiment;

[0038] Figure 3 The figure is a flowchart of an online learning algorithm in one embodiment.

[0039] Description of reference numerals:

[0040] 1-laser; 2-beam expansion lens; 3-first HWP; 4-high-power reflector; 5-first PBS; 6-reflective amplitude-type SLM; 7-second HWP; 8-first CMOS; 9-first reflective phase-type SLM; 10-second PBS; 11-second HWP; 12-second CMOS; 13-third PBS; 14-second reflective phase-type SLM; 15-third HWP; 16-fourth PBS; 17-third CMOS; 18-third reflective phase-type SLM; 19-fourth HWP; 20-fifth PBS; 21-fourth CMOS; 22-fourth reflective phase-type SLM; 23-fifth HWP; 24-sixth PBS; 25-fifth CMOS; 26-fifth reflective phase-type SLM; 27-sixth CMOS; 28-processing module; 29-control module. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0042] The same or similar numbers in the drawings of this embodiment correspond to the same or similar parts; in the description of the present invention, it should be understood that the directions or positional relationships indicated by the terms "upper", "lower", "left", "right", "inner", "outer", etc. are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limitations on this patent. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0043] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, article or device. In the absence of more restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, article or device including the element.

[0044] The embodiment of the present invention provides a spatial light modulation diffraction optical neural network online learning device, including: a laser, a beam expansion lens, a HWP, a high-power reflector, a diffraction layer, a processing module, and a control module;

[0045] The laser is used to provide a continuous and stable light source, and the light it emits is a monochromatic beam and a linearly polarized light with a fixed polarization direction;

[0046] The beam expansion lens is used to expand the monochromatic light beam, so that the light beam irradiated on the subsequent reflective SLM and CMOS is parallel light and can fill the entire modulation array;

[0047] The HWP is used to change the polarization direction of polarized light and modulate the polarization state of the light beam incident on the amplitude modulation array into p-polarized light;

[0048] The high-power reflector is used to change the light propagation direction of the light beam and emit it to the diffraction layer;

[0049] The diffraction layer is provided with several layers, which are used to obtain the final output of the diffraction optical neural network after amplitude regulation and phase modulation of the light beam;

[0050] The processing module is used to collect the output light field images of each diffraction layer of the diffraction optical neural network acquired by each CMOS and the current phase modulation parameters, and to determine the gradient to adjust the phase modulation parameters of each diffraction layer according to the set loss function;

[0051] The control module is used to load the phase modulation parameters onto the reflective phase-type SLM of the corresponding diffraction layer.

[0052] The present invention utilizes SLM to form an integrated diffraction optical neural network system, and by pre-debugging and packaging all components, the trouble of repeatedly adjusting the positions of components is avoided, thereby greatly improving the convenience and operating efficiency of the system.

[0053] Each diffraction layer includes PBS, reflective amplitude-type SLM, HWP, CMOS, and reflective phase-type SLM;

[0054] Two PBSs are provided, each for splitting the incident light into two perpendicular linearly polarized lights, wherein the p-polarized light completely passes through, and the s-polarized light is reflected at an angle of 45°, and the outgoing direction forms an angle of 90° with the p-polarized light;

[0055] The reflective amplitude-type SLM is used to perform amplitude modulation on the incident p-polarized light;

[0056] The HWP is used to change the polarization direction of the reflected light beam after the reflective amplitude-type SLM performs amplitude modulation on the light field;

[0057] The CMOS is used to obtain the light field intensity after the reflective amplitude-type SLM performs amplitude modulation;

[0058] The reflective phase-type SLM is used to phase modulate the incident p-polarized light, and serves as a diffraction layer of the diffraction optical neural network.

[0059] Furthermore, a HWP is arranged between the two PBSs.

[0060] Furthermore, a HWP is arranged between the emitting light side and the incident light side of two adjacent diffraction layers.

[0061] Furthermore, the PBS uses an adhesive to glue two right-angle prisms made of negative uniaxial crystals along the inclined surface, the optical axes of the two negative uniaxial crystals are parallel to the right-angle surface through which light passes and the incident surface, the e-light is p-polarized light, and the o-light is s-polarized light.

[0062] Exemplarily, an embodiment of the present invention provides a spatial light modulated diffraction optical neural network online learning device, comprising: a laser (1), a beam expander lens (2), a first HWP (3), a high power reflector (4), a first PBS (5), a reflective amplitude type SLM (6), a second HWP (7), a first CMOS (8), a first reflective phase type SLM (9), a second PBS (10), a second HWP (11), a second CMOS (12), a third PBS (13), a second reflective phase type SLM (14), a third HWP (15), a fourth PBS (16), a third CMOS (17), a third reflective phase type SLM (18), a fourth HWP (19), a fifth PBS (20), a fourth CMOS (21), a fourth reflective phase type SLM (22), a fifth HWP (23), a sixth PBS (24), a fifth CMOS (25), a fifth reflective phase type SLM (26), a sixth CMOS (27), a processing module (28), and a control module (29).

[0063] The laser (1) is used to provide a continuous and stable light source for the entire system, and the light emitted by the laser is monochromatic light and linearly polarized light with a fixed polarization direction;

[0064] The beam expansion lens (2) is used to expand the monochromatic light beam emitted by the laser (1), so that the energy of the light beam is dispersed to avoid damage to other components due to excessive power, while ensuring that the light beam irradiated on the reflective amplitude-type SLM (6) can fill the entire amplitude modulation array;

[0065] The first HWP (3) is used to change the polarization direction of the linearly polarized light output by the laser (1), and modulate the polarization state of the light beam incident on the amplitude modulation array into p-polarized light according to the use requirements of the reflective amplitude-type SLM (6), so as to ensure that the reflective amplitude-type SLM (6) can work normally;

[0066] The high-power reflector (4) is used to change the light propagation direction in the system to avoid the light path being too long. At the same time, since the change in the angle between the reflected light and the incident light is equal to twice the angle of rotation of the mirror surface, adding a high-power reflector (4) in the light path makes it easier to adjust the light path to collimation.

[0067] The first PBS (5) is used to split the incident light into two perpendicular linear polarized light beams, wherein the p-polarized light completely passes through, while the s-polarized light is reflected at an angle of 45°, and the outgoing direction forms an angle of 90° with the p-polarized light. Both the p-polarized light and the s-polarized light are linear polarized lights, and the polarization directions are perpendicular to each other;

[0068] The reflective amplitude-type SLM (6) is used to perform amplitude modulation on incident p-polarized light. It is composed of a sandwich structure consisting of a photoconductive layer, a dielectric reflector, a liquid crystal layer, a glass substrate transparent conductive electrode (ITO), etc., and a two-dimensional array liquid crystal panel composed of many basic independent units. The control module (29) can control the voltage to adjust the rotation direction of the liquid crystal molecules through software to achieve amplitude control of a single pixel point. The reflected light of the modulated array is s-polarized light.

[0069] The second HWP (7) is used to change the polarization direction of the reflected light beam after the reflective amplitude-type SLM (6) performs amplitude modulation on the light field, so that part of the light intensity of the modulated light field can be acquired by the subsequent CMOS through the second PBS (10) and ensure that the subsequent reflective phase-type SLM can work normally;

[0070] The first CMOS (8) is used to obtain the intensity of the light field after the reflective amplitude-type SLM (6) performs amplitude modulation;

[0071] The first reflective phase-type SLM (9) is used to perform phase modulation on incident p-polarized light, and serves as the first diffraction layer of the diffraction optical neural network. The control module (29) can control the voltage to adjust the rotation direction of the liquid crystal molecules through software, thereby achieving phase control of a single pixel point;

[0072] The second CMOS (12) is used to obtain the intensity of the light field after the first reflective phase-type SLM (9) performs phase modulation;

[0073] The second reflective phase-type SLM (14) is used to perform phase modulation on incident p-polarized light, and serves as the second diffraction layer of the diffraction optical neural network. The control module (29) can control the voltage to adjust the rotation direction of the liquid crystal molecules through software, thereby achieving phase control of a single pixel point;

[0074] The third CMOS (17) is used to obtain the intensity of the light field after the second reflective phase-type SLM (14) performs phase modulation;

[0075] The third reflective phase-type SLM (18) is used to perform phase modulation on incident p-polarized light, and serves as the third diffraction layer of the diffraction optical neural network. The control module (29) can control the voltage to adjust the rotation direction of the liquid crystal molecules through software, thereby realizing phase control of a single pixel point;

[0076] The fourth CMOS (21) is used to obtain the intensity of the light field after the third reflective phase-type SLM (18) performs phase modulation;

[0077] The fourth reflective phase-type SLM (22) is used to perform phase modulation on incident p-polarized light, and serves as the fourth diffraction layer of the diffraction optical neural network. The control module (29) can control the voltage to adjust the rotation direction of the liquid crystal molecules through software, thereby achieving phase control of a single pixel point;

[0078] The fifth CMOS (25) is used to obtain the intensity of the light field after the fourth reflective phase-type SLM (22) performs phase modulation;

[0079] The fifth reflective phase-type SLM (26) is used to perform phase modulation on incident p-polarized light, and serves as the fifth diffraction layer of the diffraction optical neural network. The control module (29) can control the voltage to adjust the rotation direction of the liquid crystal molecules through software, thereby achieving phase control of a single pixel point;

[0080] The sixth CMOS (27) is used to obtain the intensity of the light field after the fifth reflective phase-type SLM (26) performs phase modulation;

[0081] The processing module (28) is used to collect the output light field images of each layer of the diffraction optical neural network acquired by each CMOS and the current phase modulation parameters of each reflective phase-type SLM, and to calculate the gradient according to the set loss function to adjust the phase modulation parameters of each diffraction layer;

[0082] The control module (29) is used to load the phase modulation parameters calculated by the processing module (19) onto the corresponding reflective phase-type SLM.

[0083] like Figure 2 As shown, the single-pixel visible light infrared fusion compressed spectral imaging method includes the following steps:

[0084] Step 1, expanding and phase modulating the continuous and stable monochromatic linear polarized light;

[0085] Specifically, the laser emits a continuous and stable monochromatic linearly polarized laser beam, which is expanded by a beam expander lens, so that the original small-diameter beam is adjusted to a large-diameter parallel light. At this time, the incident light wave is a plane wave, and its complex amplitude is expressed as

[0086]

[0087] Since the normal operation of SLM requires the incident p-polarized light (the component of the electric vector vibrating parallel to the incident plane is called the p component, and if the linear polarized light has only the p component, it is called p-polarized light), the expanded light beam must also be polarized and modulated by the first HWP to make it p-polarized. The modulation process is to rotate the half-wave plate so that its fast axis direction forms an angle θ with the original polarization direction. Since the half-wave plate adds a phase delay π to the slow axis direction component, the linear polarized light is still linearly polarized light after passing through the half-wave plate, and the orientation of the vibration plane is rotated by 2θ compared to the incident light. Therefore, the polarization direction of the linear polarized light can be adjusted by changing the size of the angle θ to make it a continuous and stable p-polarized light.

[0088] Step 2, loading a target grayscale image on the reflective amplitude-type SLM to perform amplitude modulation on the incident p-polarized light, so that the intensity of the reflected light field is distributed according to the grayscale value of the loaded grayscale image;

[0089] Specifically, the PBS (polarization beam splitter) performs polarization filtering on the p-polarized light beam after beam expansion, which can transmit the p-polarized light and reflect the s-polarized light, so that only the p-polarized light is incident on the amplitude modulation array of the reflective amplitude-type SLM;

[0090] The working principle of PBS is to select a suitable adhesive (n e <n<n o ) Put two negative uniaxial crystals (n e <n o ) are glued along the inclined plane, and the optical axes of the two crystals are parallel to the right-angled plane through which light passes and the incident plane. At this time, the e-light is p-polarized light, and the o-light is s-polarized light (the component of the electric vector vibrating perpendicular to the incident plane is called the s component. If a line of polarized light has only the s component, it is called s-polarized light);

[0091] The working principle of amplitude-type SLM is that when no power is applied to the electrode, the arrangement of the liquid crystal is gradually twisted 90° along the direction of the light incident plane. Natural light passes through the polarizer P1 (parallel to the incident plane) to obtain polarized light with a vibration direction parallel to the incident plane, i.e., p-polarized light. After the light beam is incident on the light valve, it is reflected by the dielectric reflector and then emitted through the light valve. The polarization direction of the emitted light does not change, and no light passes through the orthogonal analyzer P2 (perpendicular to the incident plane). When the electrodes of the liquid crystal light valve are powered on, the arrangement of the liquid crystal is twisted to 45°, and the vibration direction of the emitted light reflected by the dielectric reflector is perpendicular to the incident plane. At this time, the emitted light can all pass through the analyzer P2;

[0092] According to actual use requirements, the target grayscale image is loaded on the amplitude modulation array through the control processing module. The amplitude modulation array changes the resistance of the photosensitive layer according to the grayscale value of the pixel point written into the image, so that an electric field that varies with space is formed on the liquid crystal layer. The change in the distribution of the electric field applied to the liquid crystal layer changes the distortion of the liquid crystal arrangement, so that the vibration direction of the polarized light emitted through the light valve changes, and the intensity of the light emitted through the analyzer is distributed in space, thereby achieving the purpose of spatial light modulation. At this time, the reflected light field of the reflective amplitude type SLM is the input I of the first layer of diffraction network. 1_in ,have

[0093]

[0094] Where I(x0, y0) is the grayscale value of the pixel at (x0, y0) of the written image. The light field at this time can be regarded as a linear combination of several point light sources.

[0095] Step 3, polarization modulating the reflected light of the reflective SLM again;

[0096] Specifically, since the orthogonal analyzers of the reflective amplitude-type SLM are perpendicular to the incident surface direction, the reflected light is s-polarized;

[0097] When the reflected light passes through the PBS, since the PBS transmits p-polarized light and reflects s-polarized light, the s-polarized light reflected by the reflective amplitude-type SLM is reflected by the PBS in a direction perpendicular to the original incident light path;

[0098] The fast axis direction of the second HWP is then adjusted to a position approximately 43° with the vibration direction of the electric vector of the light wave. After the light wave is polarized by the second HWP, the vibration direction of the electric vector of the light wave is deflected by 86°, so that the p component of the amplitude modulated light beam is approximately 99.7564% of the original amplitude, and the s component is approximately 6.9756% of the original amplitude.

[0099] Step 4, the polarization modulated light beam is split by the PBS, the p component is incident on the reflective phase-type SLM along the original optical path, and the s component is reflected onto the CMOS;

[0100] Specifically, the polarization modulated light beam is split by the PBS, and the p component of the light beam is incident on the phase modulation array of the reflective phase-type SLM along the original optical path, while the s component is reflected to the CMOS array. At this time, the ratio of the light intensity of the reflective phase-type SLM to the light intensity on the CMOS is about 204.5:1. A small part of the light intensity is reflected to the CMOS, while most of the light intensity continues to maintain the diffraction optical neural network.

[0101] Phase-type spatial light modulators use the properties of optical anisotropic materials to control the phase of incident light. In liquid crystals, the orientation of molecules is affected by the external electric field, which in turn changes the propagation characteristics of light waves. When the polarization direction of the incident light is parallel to the optical axis of the liquid crystal molecules, the liquid crystal is transparent to o-light, but produces a phase delay to e-light. By applying a pixel-level voltage to the liquid crystal, the orientation of the liquid crystal molecules can be controlled, thereby changing the optical path of the e-light and fine-tuning the phase of the light wave.

[0102] The working principle of CMOS camera is based on CMOS image sensor. Each pixel contains a photosensitive charge-coupled device, which generates charges when photons hit these photosensitive areas. These charges are converted and amplified and stored in adjacent capacitors, and then transferred to analog-to-digital converters (ADCs) by row through column-select devices for digitization, ultimately forming digital image output.

[0103] After that, each reflective phase-type SLM outputs s-polarized light, which is reflected by the PBS and modulated by the second HWP to have a linear polarization angle, and then split by the PBS. Each reflective phase-type SLM in the phase modulation process serves as a diffraction layer, and the number of reflective phase-type SLMs can be changed according to actual work needs.

[0104] Each CMOS will obtain a small portion of the light intensity of its corresponding reflective phase-type SLM after polarization modulation by the second HWP, which provides support for the processing module to calculate the loss value and gradient.

[0105] Step 5, see Figure 3 In the flowchart of an online learning algorithm in one embodiment shown, the processing module calculates the loss value and gradient of the model according to a set loss function, and the control module continuously adjusts the phase modulation parameters according to the calculated gradient until the loss value meets the set conditions;

[0106] Specifically, the processing module is composed of a graphics processing unit (GPU). After initializing the network parameters, it accepts new data and signals and can calculate the gradient information according to the set loss function. The network parameters are updated with the calculated gradient information and passed to each reflective phase-type SLM through the control module.

[0107] In some embodiments, if negative Pearson correlation coefficient (PCC) is used as the loss function, the specific process is:

[0108] Calculate the output of the entire spatial light modulation diffraction optical neural network, that is, the covariance between the image O acquired by the sixth CMOS and the true label T (ground truth);

[0109] Calculate the standard deviation of 0 and T;

[0110] Divide the covariance by the product of the standard deviations of O and T to obtain the Pearson correlation coefficient;

[0111] Take the negative value as the loss function, that is, L(O,T) = -PCC(O,T). From this, we can get the expression:

[0112]

[0113] in, and Respectively represent the mean of the two.

[0114] The control module continuously adjusts the phase modulation parameters loaded on each reflective phase-type SLM according to the calculated gradient until the loss value is less than the set threshold. The specific steps are as follows:

[0115] 1. Call the required extension library;

[0116] 2. Check if there is an available GPU;

[0117] 3. Set the connection port, baud rate, timeout duration and other parameters according to the actual serial port of the phase-type SLM;

[0118] 4. Set the connection port, baud rate, timeout duration and other parameters according to the actual serial port of CMOS;

[0119] 5. Initialize the phase modulation parameters as PyTorch tensors and move them to the processing module, i.e., the GPU;

[0120] 6. Define the function that deploys the phase modulation parameters to a specific SLM;

[0121] 7. Define the function to capture the image from the CMOS camera;

[0122] 8. Define PCC function;

[0123] 9. Define a function to update the phase modulation parameters of all deployed phase-type SLMs;

[0124] 10. Define the online learning training loop function, specifically:

[0125] 1) Set up the PyTorch optimizer to update the calculated phase modulation parameters;

[0126] 2) Capturing the result image through CMOS and setting the image acquired by the sixth CMOS to 0;

[0127] 3) Set or read the target image as T according to the specific task;

[0128] 4) Calculate the loss;

[0129] 5) Check whether the loss reaches the threshold;

[0130] 6) If the threshold is not reached, the phase modulation parameters are calculated using the optimizer and the phase modulation parameters of all deployed SLMs are updated, and then step 1) is restarted;

[0131] 7) If the threshold is reached, stop the loop.

[0132] 11. Define a function to detect whether the amplitude-type SLM deployment is updated, and start online learning training when the amplitude-type SLM deployment is updated.

[0133] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention.

Claims

1. A spatial light modulation diffraction optical neural network online learning device, characterized in that: include: Laser, beam expansion lens, HWP, high-power reflector, diffraction layer, processing module, control module; The laser is used to provide a continuous and stable light source, and the light it emits is a monochromatic beam and a linearly polarized light with a fixed polarization direction; The beam expansion lens is used to expand the monochromatic light beam, so that the light beam irradiated on the subsequent reflective SLM and CMOS is parallel light and can fill the entire modulation array; The HWP is used to change the polarization direction of polarized light and modulate the polarization state of the light beam incident on the amplitude modulation array into p-polarized light; The high-power reflector is used to change the light propagation direction of the light beam and emit it to the diffraction layer; The diffraction layer is provided with several layers, which are used to obtain the final output of the diffraction optical neural network after amplitude regulation and phase modulation of the light beam; The processing module is used to collect the output light field images of each diffraction layer of the diffraction optical neural network acquired by each CMOS and the current phase modulation parameters, and to determine the gradient to adjust the phase modulation parameters of each diffraction layer according to the set loss function; The control module is used to load the phase modulation parameters onto the reflective phase-type SLM of the corresponding diffraction layer.

2. The spatial light modulation diffraction optical neural network online learning device according to claim 1 is characterized in that: Each diffraction layer includes PBS, reflective amplitude-type SLM, HWP, CMOS, and reflective phase-type SLM; Two PBSs are provided, each for splitting the incident light into two perpendicular linearly polarized lights, wherein the p-polarized light completely passes through, and the s-polarized light is reflected at an angle of 45°, and the outgoing direction forms an angle of 90° with the p-polarized light; The reflective amplitude-type SLM is used to perform amplitude modulation on incident p-polarized light; The HWP is used to change the polarization direction of the reflected light beam after the reflective amplitude-type SLM performs amplitude modulation on the light field; The CMOS is used to obtain the intensity of the light field after the reflective amplitude-type SLM performs amplitude modulation; The reflective phase-type SLM is used to phase modulate the incident p-polarized light, and serves as a diffraction layer of the diffraction optical neural network.

3. The spatial light modulation diffraction optical neural network online learning device according to claim 2 is characterized in that: A HWP is provided between the two PBSs.

4. The spatial light modulation diffraction optical neural network online learning device according to claim 3 is characterized in that: A HWP is arranged between the light emitting side and the light incident side of two adjacent diffraction layers.

5. The spatial light modulation diffraction optical neural network online learning device according to any one of claims 1 to 4, characterized in that: The PBS uses an adhesive to glue two right-angle prisms made of negative uniaxial crystals along the inclined surface. The optical axes of the two negative uniaxial crystals are parallel to the right-angle surface through which light passes and the incident surface. The e light is p-polarized light and the o light is s-polarized light.

6. An imaging method used in the spatial light modulation diffraction optical neural network online learning device according to any one of claims 1 to 5, characterized in that: The method includes: Expand and phase modulate continuous and stable monochromatic linear polarized light; The target grayscale image is loaded on the reflective amplitude-type SLM to perform amplitude modulation on the incident p-polarized light, so that the intensity of the reflected light field is distributed according to the grayscale value of the loaded grayscale image; The reflected light of the reflective SLM is polarized again; After polarization modulation again, the light beam is split by the PBS, and the p component is incident on the reflective phase-type SLM along the original optical path, and the s component is reflected onto the CMOS; The processing module calculates the loss value and gradient of the model according to the set loss function, and the control module continuously adjusts the phase modulation parameters according to the gradient until the loss value meets the set conditions and outputs the final image output.

7. The imaging method according to claim 6, characterized in that: The method of expanding and phase modulating the continuous and stable monochromatic linearly polarized light specifically includes: the laser emits a continuous and stable monochromatic linearly polarized laser beam, which is expanded by a beam expander lens so that the original small-diameter beam is adjusted to a large-diameter parallel light. The incident light wave is a plane wave, and its complex amplitude is expressed as HWP performs polarization modulation on the expanded light beam to make it p-polarized light. The modulation process is to rotate the half-wave plate so that its fast axis direction forms an angle θ with the original polarization direction, and add a phase delay of π. After the linearly polarized light passes through the half-wave plate, it is still linearly polarized light and the orientation of the vibration plane is rotated by 2θ compared to the incident light.

8. The imaging method according to claim 6 or 7, characterized in that: The method of loading a target grayscale image on the reflective amplitude-type SLM to amplitude modulate the incident p-polarized light so that the intensity of the reflected light field is distributed according to the grayscale value of the loaded grayscale image, specifically includes: PBS performs polarization filtering on the p-polarized light beam after beam expansion, which can transmit p-polarized light and reflect s-polarized light, so that only p-polarized light is incident on the amplitude modulation array of the reflective amplitude-type SLM.

9. The imaging method according to claim 8, characterized in that: The reflected light field of the reflective amplitude-type SLM is the input I of the first diffraction network. 1_in , Where I(x0, y0) is the grayscale value of the pixel at (x0, y0) of the written image. The light field at this time can be regarded as a linear combination of several point light sources.

Images