Optoelectronic module using smart pixel optical modulator, and optical neural network computer comprising same

Abstract

An optoelectronic module using a smart pixel optical modulator comprises: a plurality of first light sources spaced a predetermined distance apart from each other; a plurality of first lenses corresponding to each of the plurality of first light sources, and reducing the divergence angle of a light beam emitted from the corresponding first light source; a plurality of smart pixel optical modulators corresponding to each of the plurality of first lenses, and adjusting the intensity of a light beam that passed through a corresponding first lens through respective pixels having a weight stored in a memory; a plurality of photodetectors which are spaced apart from the plurality of smart pixel optical modulators, and which acquire current values according to the intensity of light; and a lens unit formed between the plurality of smart pixel optical modulators and the plurality of photodetectors so as to collect, into one photodetector, light beams emitted from different first light sources passing through pixels at the same relative position of each smart pixel optical modulator.

Description

Optoelectronic module using smart pixel optical modulator and optical neural network computer including the same

[0001] The present invention relates to an optoelectronic module using a smart pixel optical modulator and an optical neural network computer including the same, and more particularly, to an optoelectronic module capable of two-way redundant communication to replace a telecommunication line, and capable of performing high-speed calculations in parallel without complex electronic circuits by combining a light source, a lens, a smart pixel optical modulator, and a light detector, and an optical convolutional neural network computer including the same.

[0002] Recently, in fields such as image processing, the demand for high-speed computational processing technology targeting two-dimensional and three-dimensional information has been rapidly increasing.

[0003] An optical computer is a computer that uses optical integrated circuits (ICs) that utilize the properties of light in its computational circuits, allowing for more redundant communication than telecommunication lines.

[0004] Current electronic calculators are comprised of electronic circuits. Consequently, computational speeds are limited by factors such as component delays and stray capacitance, limiting their potential for high-speed computing. Optical computers are expected to overcome these limitations.

[0005] Figure 1 shows an example of a conventional neural network. Input values , Weighted in Multiply by the bias This method calculates the output value by adding the values ​​and then applying the sigmoid function б, and uses this as the input value of the next layer.

[0006] In general, the equation connecting the (l-1)th input layer and the lth output layer is as follows: Mathematical Equation 1. Here, Nl-1 represents the number of artificial neurons or nodes in the (l-1)th layer.

[0007] [Mathematical Formula 1]

[0008]

[0009] The conventional way to calculate neural networks is to use a digital computer to calculate mathematical expression 1.

[0010] Figure 2 is a conceptual diagram of a typical convolutional neural network. As illustrated in Figure 2, a conventional CNN (Convolutional Neural Network) consists of an input image and a series of layers. In each layer, a series of kernels perform convolution with the input image, and the output of the convolution is pooled and passed to the next layer after a pooling operation.

[0011] In the conventional method of implementing optical CNNs to implement optically implemented convolution operations, the function of the optical convolution layer is implemented using a method of implementing a complex-valued Fourier filter.

[0012] Figure 3 is a conceptual diagram of a conventional 4f correlation optical convolutional neural network. Referring to Figure 3, the optical CNN architecture utilizes a 4f correlation design.

[0013] A traditional 4f correlation consists of two lenses, which have the same focal length, and the input and output planes can be located at the focal planes of the first and second lenses, respectively.

[0014] Meanwhile, using the Fresnel approximation, the lens provides a Fourier transform relationship between the electric field distributions located in the front focal plane and the back focal plane, and when two lenses are placed in series, it provides a Fourier transform that reverses the sign of the input electric field.

[0015] At this time, if a mask is placed on the plane between the two lenses, arbitrary Fourier domain filtering operations can be implemented, including low pass, high pass, band stop, band reject, and filtering operations, and more complex filtering is possible if a complex-valued transmittance mask is used on the filtering plane.

[0016] Here, the configuration of the 4f correlation array is configured so that the optical CNN architecture computes convolution operations on multiple input values ​​in parallel with one mask layer.

[0017] Figure 4 is a conceptual diagram of a conventional parallel optical convolutional neural network module. Referring to Figure 4, a convolution operation calculated in parallel is implemented using a system that adds two aligned lenslet array stacks and a filter mask array.

[0018] To implement a parallel photosynthetic neural network module like that shown in Figure 5, planar optics based on diffractive optical elements or metasurfaces are used. Each lens has a separate mask, allowing for parallel processing of different filtering functions.

[0019] To transmit and post-process images in an optical CNN system, an array of lenslets, an array of aligned sources, and an array of sensors that collect the output from the 4f correlations of each lenslet stack are required.

[0020] In the 4f correlation, the space-bandwidth product is can be given as, where D is the aperture size, f is the focal length, represents the optical wavelength.

[0021] For example, D is 0.6 mm, f is 3 mm, For a system with a resolution of 500 nm, the spatial bandwidth product is approximately 240х240, which means that the system can reliably compute approximately 240х240 input or output pixels.

[0022] Fig. 5 is the input value Weighted in Bias by multiplying This is a conceptual diagram of a conventional photosynthetic neural network module, which is a structure that calculates the output value by adding σ and then applying the sigmoid function σ, and uses this as the input value of the next layer.

[0023] From the input node's perspective, this is a method of calculating a linear combination by multiplying the weight corresponding to the kernel from one input node and sending it to multiple output nodes in the vicinity, and from the output node's perspective, it is a method of receiving values ​​from multiple input nodes in the vicinity, multiplying them by the weight, and then calculating a linear combination.

[0024] Although Figure 5 shows four inputs and four outputs as an example, the number of inputs and outputs is not limited to this.

[0025] The conventional method of calculating a neural network uses a digital computer, and can be expressed as in mathematical equation 2.

[0026] [Equation 2]

[0027]

[0028]

[0029] Here, N represents the number of elements in the input value, and Nm represents the number of elements in the convolution kernel or filter.

[0030] This shows the case where the input / output and kernel are one-dimensional arrays, but it can be extended and used in the same way for two-dimensional arrays.

[0031] Meanwhile, when implementing an analog neural network electrically and simultaneously performing calculations, problems arise, such as electromagnetic noise due to overlapping wires and complex circuit wiring layout. To address these issues, a previous patent (Patent Document 4) proposed an optoelectronic module using a spatial light modulator and an optical computer including the same.

[0032] The optical computer architecture presented in patent document 4 enables parallel calculations corresponding to the number of pixels of a spatial light modulator compared to an electronic computer, thereby increasing the processing capacity by 10 when using a Full HD LCD microdisplay. 6 This has the effect of increasing the resolution by about 10 times, and when using a micro-optical system such as a micro-lens array, it offers the advantage of miniaturizing the system to a size of about 10 mm × 10 mm × 10 mm. However, one drawback is that the direction of light propagation is fixed from the first substrate to the second substrate, making it difficult to move data in the opposite direction.

[0033] In neural network calculations, when weights are determined, data starts at the input layer and moves in one direction to the next layer, so existing optical computer structures can process data at a very high speed, which is not a problem. This process is called inference, and it can be viewed as a case where the neural network weights have already been determined and image processing is performed on various input images. However, in the case of training calculations that require finding the weights themselves, a backpropagation algorithm must be used, but it is not easy to perform this backpropagation algorithm with the structure of patent document 4.

[0034] [Prior Art Literature]

[0035] [Patent Document]

[0036] (Patent Document 1) KR 10-0624852 B1

[0037] (Patent Document 2) KR 10-2079833 B1

[0038] (Patent Document 3) JP 6746074 B2

[0039] (Patent Document 4) KR 10-2622313 B1

[0040] (Reference 1) Ju, Y.-G. Bidirectional Optical Neural Networks Based on Free-Space Optics Using Lens Arrays and Spatial Light Modulator. Micromachines 2024, 15, 701. https: / doi.org / 10.3390 / mi15060701

[0041] The technical problem to be achieved by the present invention is conceived from this point of view, and its purpose is to provide an optoelectronic module using a smart pixel optical modulator with a fast switching speed.

[0042] In addition, another purpose is to provide an optoelectronic module using a smart pixel optical modulator that can process an input / output pixel array of a much larger size than conventional systems by utilizing the characteristics of light instead of complex electronic circuits, and an optical convolutional neural network computer including the same.

[0043] In addition, the purpose is to provide a bidirectional optoelectronic module using a smart pixel optical modulator having a fast switching speed and an optical neural network computer including the same, and further to provide an optoelectronic module using a smart pixel optical modulator capable of bidirectional optical connection between substrates of optical neural network hardware and an optical neural network computer including the same.

[0044] The technical problems to be solved by the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0045] In order to achieve the above technical problem, an optoelectronic module using a smart pixel optical modulator according to an embodiment of the present invention includes: a plurality of first light sources formed to be spaced apart from each other by a predetermined distance; a plurality of first lenses formed to correspond to the plurality of first light sources, respectively, and to reduce a divergence angle of light emitted from the corresponding first light sources; a plurality of smart pixel optical modulators formed to correspond to the plurality of first lenses, respectively, and to adjust the intensity of light passing through the corresponding first lens through respective pixels having weights stored in a memory; a plurality of photodetectors formed to be spaced apart from the plurality of smart pixel optical modulators and to obtain a current value according to the intensity of light; and a lens unit formed between the plurality of smart pixel optical modulators and the plurality of photodetectors, and collecting light emitted from different first light sources passing through pixels at the same relative positions of the respective smart pixel optical modulators onto one photodetector.

[0046] According to another embodiment of the present invention for realizing the above-described object, an optoelectronic module using a smart pixel optical modulator comprises: a plurality of electrical input nodes formed at a predetermined distance from each other; a plurality of smart pixel optical modulators, each of which is composed of a plurality of pixels and connects an input signal input through the electrical input nodes to each of the pixels through an electrical fan-out structure; a plurality of photodetectors formed at a distance from the plurality of smart pixel optical modulators and obtaining a current value according to the intensity of light; and a lens unit formed between the plurality of smart pixel optical modulators and the plurality of photodetectors and collecting light rays originating from different smart pixel optical modulators passing through pixels at the same relative positions of the respective smart pixel optical modulators onto a single photodetector.

[0047] In order to achieve the above object of the present invention, according to another embodiment of the present invention, an optoelectronic module using a smart pixel optical modulator comprises: a plurality of first light sources formed at a predetermined distance from each other; a plurality of first lenses formed corresponding to the plurality of first light sources, respectively, and reducing a divergence angle of light emitted from the corresponding first light sources; a plurality of smart pixel optical modulators formed corresponding to the plurality of first lenses, respectively, and controlling the intensity of light passing through the corresponding first lens through respective pixels having weights stored in a memory; a plurality of first photodetectors formed spaced apart from the plurality of smart pixel optical modulators and obtaining a current value according to the intensity of light; and a lens unit formed between the plurality of smart pixel optical modulators and the plurality of first photodetectors, the lens unit collecting light emitted from different first light sources passing through pixels at different relative positions of each smart pixel optical modulator onto a single first photodetector, wherein the lens unit comprises: a plurality of second lenses formed corresponding to the plurality of smart pixel optical modulators, respectively, and collecting light into a single point; and a plurality of third lenses corresponding to each of the second lenses, each of which passes light passing through the second lens.

[0048] According to another embodiment of the present invention for achieving the above object, an optoelectronic module using a smart pixel optical modulator comprises: a plurality of electrical input nodes formed at a predetermined distance from each other; a plurality of smart pixel optical modulators, each of which is composed of a plurality of pixels and connects an input signal input through the electrical input nodes to each of the pixels through an electrical fan-out structure; a plurality of first photodetectors formed at a distance from the plurality of smart pixel optical modulators and obtaining a current value according to the intensity of light; and a lens formed between the plurality of smart pixel optical modulators and the plurality of first photodetectors and collecting light rays originating from different smart pixel optical modulators passing through pixels at different relative positions of each smart pixel optical modulator onto a single photodetector.

[0049] In order to achieve the above object, according to another embodiment of the present invention, a bidirectional optoelectronic module using a smart pixel optical modulator comprises: a plurality of first light sources formed at a predetermined distance from each other; a plurality of first lenses formed corresponding to the plurality of first light sources, respectively, and reducing a divergence angle of light emitted from the corresponding first light sources; a plurality of smart pixel optical modulators formed corresponding to the plurality of first lenses, respectively, and adjusting the intensity of light passing through the corresponding first lens through respective pixels having weights stored in a memory; a plurality of first photodetectors formed spaced apart from the plurality of smart pixel optical modulators, the first photodetectors obtaining a current value according to the intensity of light; a lens unit formed between the plurality of smart pixel optical modulators and the plurality of first photodetectors, the lens unit collecting light emitted from different first light sources passing through pixels at the same relative positions of the respective smart pixel optical modulators onto one of the plurality of first photodetectors; a plurality of third light sources formed at a predetermined distance from the plurality of first photodetectors, the third light sources allowing light to travel in a direction opposite to the first light sources; And a plurality of third light detectors formed at a predetermined distance from the first light source, collecting light rays irradiated from the plurality of third light sources and passing through the smart pixel light modulator and the first lens.

[0050] According to another embodiment of the present invention for achieving the above object, a bidirectional optoelectronic module using a smart pixel optical modulator comprises: a plurality of electrical input / output nodes formed at a predetermined distance from each other; a plurality of smart pixel optical modulators formed of a plurality of pixels and connecting an input signal input through the electrical input / output nodes to each of the pixels through an electrical fan-out structure; a plurality of photodetectors formed at a distance from the plurality of smart pixel optical modulators and obtaining a current value according to the intensity of light; a lens unit formed between the plurality of smart pixel optical modulators and the plurality of photodetectors and collecting light rays originating from different smart pixel optical modulators departing from pixels at the same relative positions of each smart pixel optical modulator onto a single photodetector; And a plurality of light sources formed at a predetermined distance from the plurality of light detectors and through which light propagates in a direction opposite to the input signal; wherein the smart pixel optical modulator comprises a plurality of light detectors that collect light emitted from the plurality of light sources and passed through the lens unit, an electronic processing unit connected to the light detectors, and an electrical fan-in structure that connects the electrical input / output node and the plurality of pixels.

[0051] According to the optoelectronic module using the smart pixel optical modulator and the optical computer including the same, an analog / digital optical computer structure is implemented using optical connections instead of purely electronic circuits, so that even if the number of input and output terminals increases to N and M, respectively, the connection does not become complicated or noise is generated as in electronic circuits. Accordingly, by performing approximately NXM multiplications and additions at once, important part calculations operate in parallel, enabling ultra-small and ultra-high-speed calculations.

[0052] In addition, the smart pixel light modulator receives an input signal through a photodetector or electrical input node in the smart pixel, amplifies the input signal by the weight stored in the memory by the electronic circuit, and then outputs the light source intensity proportional to it. Therefore, the switching speed is very fast, at hundreds of MHz or more, compared to the existing spatial light modulator (LCD, DMD). In addition, the smart pixel light modulator can perform various operations because it can perform optical input-optical output or various combinations of opto-electronic input / output like the existing spatial light modulator, and can perform operations such as multiplication and addition using the values ​​stored in the memory and the input signal.

[0053] Since the speed of the smart pixel light modulator is much faster than that of a conventional spatial light modulator, this structure can significantly improve parallel processing speed by replacing the spatial light modulator in algorithms that require rapid weight updates, such as bidirectional optical neural networks or backpropagation algorithms.

[0054] The effects of the present invention are not limited to the effects described above, and should be understood to include all effects that can be inferred from the detailed description of the present invention or the composition of the invention described in the claims.

[0055] Figure 1 is a conceptual diagram of a conventional neural network.

[0056] Figure 2 is a conceptual diagram of a general convolutional neural network.

[0057] Figure 3 is a conceptual diagram of a conventional 4f correlation optical convolutional neural network.

[0058] Figure 4 is a conceptual diagram of a conventional parallel photosynthetic neural network module.

[0059] Figure 5 is a conceptual diagram of a conventional photosynthetic neural network module.

[0060] FIG. 6 is a conceptual diagram of an optoelectronic module using a spatial light modulator according to the first embodiment of the present invention.

[0061] Figure 7 is a conceptual diagram showing the correlation between the optoelectronic module of Figure 6 and the neural network.

[0062] Fig. 8 is a drawing showing an example of cascading the optoelectronic modules of Fig. 6.

[0063] Figure 9 is a conceptual diagram of an optoelectronic module using a spatial light modulator according to a second embodiment of the present invention.

[0064] Fig. 10 is a drawing showing an example of cascading the optoelectronic modules of Fig. 9.

[0065] Fig. 11 is a conceptual diagram of an optoelectronic module using a spatial light modulator according to a third embodiment of the present invention.

[0066] FIG. 12 is a conceptual diagram of an optical neural network computer using a smart pixel optical modulator according to the first embodiment of the present invention.

[0067] FIG. 13 is a conceptual diagram of an optical neural network computer using a smart pixel optical modulator including a fan-out according to a second embodiment of the present invention.

[0068] Fig. 14 is a conceptual diagram of the smart pixel optical modulator of Fig. 13.

[0069] FIG. 15 is a conceptual diagram of an optical neural network computer using a smart pixel optical modulator according to a third embodiment of the present invention.

[0070] Fig. 16 is a conceptual diagram of an optoelectronic module using a spatial light modulator according to the fourth embodiment of the present invention.

[0071] Fig. 17 is a three-dimensional example of an optoelectronic module using the spatial light modulator of Fig. 16.

[0072] Fig. 18 is an example diagram of the pixel spacing of the spatial light modulator of Fig. 17.

[0073] Fig. 19 is a conceptual diagram of a multiple output optoelectronic module using the spatial light modulator of Fig. 16.

[0074] FIG. 20 is a conceptual diagram of an optoelectronic module using a smart pixel optical modulator according to a fifth embodiment of the present invention.

[0075] FIG. 21 is a conceptual diagram of an optoelectronic module using a smart pixel optical modulator including a fan-out according to a sixth embodiment of the present invention.

[0076] Fig. 22 is a conceptual diagram of the smart pixel optical modulator of Fig. 21.

[0077] Fig. 23 is a conceptual diagram of a bidirectional optoelectronic module using a spatial light modulator according to the first embodiment of the present invention.

[0078] Fig. 24 is a conceptual diagram of a smart pixel optical modulator applied to the bidirectional optoelectronic module of Fig. 23.

[0079] FIG. 25 is a conceptual diagram of a bidirectional optoelectronic module using a smart pixel optical modulator including fan-in and fan-out according to a second embodiment of the present invention.

[0080] Fig. 26 is a conceptual diagram of the smart pixel optical modulator of Fig. 25.

[0081] Figure 27 is a conceptual diagram explaining the conventional backpropagation algorithm in a neural network.

[0082] Fig. 28 is a conceptual diagram of a non-differential mode bidirectional optoelectronic module using a spatial light modulator according to a third embodiment of the present invention.

[0083] Figure 29 is a conceptual diagram of the third light source of Figure 28 implemented using a diffraction grating and a prism.

[0084] Fig. 30 is a drawing showing an example of cascading the optoelectronic modules of Fig. 28.

[0085] Fig. 31 is a conceptual diagram of a differential mode bidirectional optoelectronic module using a spatial light modulator according to the fourth embodiment of the present invention.

[0086] Fig. 32 is a drawing showing an example of cascading the optoelectronic modules of Fig. 31.

[0087] FIG. 33 is a conceptual diagram of a bidirectional optoelectronic module using a smart pixel optical modulator according to a fifth embodiment of the present invention.

[0088] Fig. 34 is a conceptual diagram of the smart pixel optical modulator of Fig. 33.

[0089] FIG. 35 is a conceptual diagram of a bidirectional optoelectronic module using a smart pixel optical modulator including fan-in and fan-out according to a sixth embodiment of the present invention.

[0090] Fig. 36 is a conceptual diagram of the first smart pixel optical modulator of Fig. 35.

[0091] Figure 37 is a conceptual diagram of implementing input / output node scale-up in an optical neural network computer using a smart pixel optical modulator.

[0092] The present invention is susceptible to various modifications and variations, and specific embodiments are illustrated in the drawings and described in detail in the text. However, this is not intended to limit the present invention to a specific disclosed form, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention. Throughout the description of each drawing, similar reference numerals have been used to designate similar components.

[0093] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0094] Hereinafter, with reference to the attached drawings, a preferred embodiment of the present invention will be described in more detail.

[0095] FIG. 6 is a conceptual diagram of an optoelectronic module using a spatial light modulator according to the first embodiment of the present invention.

[0096] The optoelectronic module (1010) using a spatial light modulator according to the present invention operates in parallel by combining a light source, a lens, a spatial light modulator, a light detector, etc. instead of a complex electronic circuit. The optoelectronic module (1010) according to the present invention can be applied to an optical computer.

[0097] Referring to FIG. 6, an optoelectronic module (1010) according to a first embodiment of the present invention includes a plurality of first light sources (1101a, 1101b), a plurality of first lenses (1102a, 1102b), a plurality of spatial light modulators (1103a, 1103b), a plurality of photodetectors (1120a, 1120b, 1120c, 1120d), and a lens unit (1104a, 1104b, 1105).

[0098] A plurality of first light sources (1101a, 1101b) may be formed at regular intervals on the first substrate (1100), but may also be formed on different substrates.

[0099] The optical optoelectronic module (1010) according to the first embodiment of the present invention may further include a plurality of electronic processing units (1121a, 1121b, 1121c, 1121d) and a plurality of second light sources (1122a, 1122b, 1122c, 1122d).

[0100] A plurality of photodetectors (1120a, 1120b, 1120c, 1120d), a plurality of electronic processing units (1121a, 1121b, 1121c, 1121d), and a plurality of second light sources (1122a, 1122b, 1122c, 1122d) may be formed on a second substrate (1130), but may alternatively be formed separately on another substrate, or one or more elements may be formed on the same substrate. For example, the first substrate (1100) and the second substrate (1130) may be semiconductor substrates or PCBs (Printed Circuit Boards).

[0101] In FIG. 6, the first light source (1101a, 1101b), the first lens (1102a, 1102b), the spatial light modulator (1103a, 1103b), etc. are expressed as two, and the photodetector (1120a, 1120b, 1120c, 1120d), the electronic processing unit (1121a, 1121b, 1121c, 1121d), and the second light source (1122a, 1122b, 1122c, 1122d) are expressed as four, but these are only examples, and the number of each element can be changed and designed as needed.

[0102] For example, in the present invention, the light source may be a light emitting diode (LED) or a semiconductor laser, particularly a vertical-cavity surface-emitting laser. The intensity of light can be controlled by varying the intensity of the current flowing through the light source, thereby individually controlling the input values.

[0103] Light emitted from a plurality of first light sources (1101a, 1101b) passes through a plurality of corresponding first lenses (1102a, 1102b), and then passes through a corresponding spatial light modulator (1103a, 1103b). The first lenses (1102a, 1102b) reduce the divergence angle of the light emitted from the corresponding first light sources (1101a, 1101b), and preferably modulate the light into parallel light.

[0104] In Fig. 6, four representative light rays are shown among the light emitted from each first light source (1101a, 1101b), and the four light rays pass through individual pixels in the spatial light modulator (1103a, 1103b).

[0105] The spatial light modulators (1103a, 1103b) are formed to correspond to each of the plurality of first lenses (1102a, 1102b), and adjust the intensity of light passing through the corresponding first lenses (1102a, 1102b) through each pixel having a preset weight.

[0106] For example, liquid crystal display panels (LCDs) can be used as each of the spatial light modulators (1103a, 1103b). Since the LCD panel regulates the intensity of incoming light to different transmittances according to voltage and transmits the light, the intensity of each light beam can be adjusted to a predetermined value.

[0107] Therefore, even if the light originates from the same light source, the intensity of the light passing through each pixel on the LCD panel varies depending on the transmittance of each pixel. For example, if the illuminance of the light source is 1.0 W / m2, if the transmittance of the LCD pixel is 0.10, the illuminance of the output light is 0.10 W / m2, and if the transmittance is 0.05, the illuminance of the output light is 0.05 W / m2.

[0108] The transmittance of the LCD panel can be adjusted in more than 256 steps depending on the voltage, so in this way, the product of the weights and input values ​​in the neural network can be implemented. Here, the input value is the intensity of the light source, and the transmittance of the spatial light modulator corresponds to the weight. In addition, the number of pixels of the spatial light modulator can be adjusted to be equal to the number of output nodes.

[0109] Light rays passing through the spatial light modulators (1103a, 1103b) are input to the lens units (1104a, 1104b, 1105). The lens units (1104a, 1104b, 1105) are formed between a plurality of spatial light modulators (1103a, 1103b) and a plurality of photodetectors (1120a, 1120b, 1120c, 1120d).

[0110] Additionally, the lens units (1104a, 1104b, 1105) serve to collect light emitted from different first light sources (1101a, 1101b) passing through pixels at the same relative positions of each spatial light modulator (1103a, 1103b) onto one photodetector (1120a, 1120b, 1120c, 1120d).

[0111] In the first embodiment of the present invention, the lens units (1104a, 1104b, 1105) are formed corresponding to the plurality of spatial light modulators (1103a, 1103b), respectively, and include a plurality of second lenses (1104a, 1104b) that focus light rays to a single point, and a third lens (1105) that passes all light rays that have passed through the plurality of second lenses (1104a, 1104b). If the light rays incident on the second lenses (1104a, 1104b) are parallel light rays, they are focused at a focus.

[0112] The light rays passing through each spatial light modulator (1103a, 1103b) pass through the corresponding second lens (1104a, 1104b), are focused, and then spread out again, where they pass through the third lens (1105) again.

[0113] When a ray passes through the second lens (1104a, 1104b), the angle of the ray changes depending on the height of the ray away from the optical axis of the second lens (1104a, 1104b). The ray (1110a) passing through the highest position on the optical axis of the second lens (1104a) and the ray (1110b) passing through the highest position on the optical axis of the second lens (1104b) have the same angle with respect to the horizontal axis.

[0114] The two light rays (1110a, 1110b) enter the third lens (1105) while forming parallel light, and thus converge at the same point on the focal plane of the third lens (1105). In Fig. 6, the two light rays converge on one photodetector (1120d).

[0115] Similarly, light rays (1111a) passing through the second lens (1104a) and light rays (1111b) passing through the second lens (1104b), which are rays at relatively the same position, pass through the third lens (1105) and are then gathered on one photodetector (1120c).

[0116] In addition, light rays (1112a) passing through the second lens (1104a) and light rays (1112b) passing through the second lens (1104b) pass through the third lens (1105) and are then collected on one photodetector (1120b), and light rays (1113a) passing through the second lens (1104a) and light rays (1113b) passing through the second lens (1104b) pass through the third lens (1105) and are then collected on one photodetector (1120a).

[0117] The photodetectors (1120a, 1120b, 1120c, 1120d) can usually be composed of optical semiconductor devices such as photodiodes. The current flowing through the photodetectors (1120a, 1120b, 1120c, 1120d) is ultimately the sum of the intensities of light arriving at each photodetector (1120a, 1120b, 1120c, 1120d). , It has a value proportional to .

[0118] Fig. 7 shows a diagram comparing the input / output of an optical computer and a neural network. The electronic processing units (1121a, 1121b, 1121c, 1121d) add the current value or the amplified value thereof obtained from the corresponding photodetectors (1120a, 1120b, 1120c, 1120d) and the bias values ​​(b3, b2, b1, b0) stored in the electronic processing units (1121a, 1121b, 1121c, 1121d), and then calculate the sigmoid function b using the values ​​as input, and obtain the neural network output value.

[0119] In one embodiment, each output value of the electronic processing units (1121a, 1121b, 1121c, 1121d) may be transmitted to each of the connected second light sources (1122a, 1122b, 1122c, 1122d). Accordingly, each of the output values ​​of the electronic processing units (1121a, 1121b, 1121c, 1121d) , , , can be adjusted proportionally.

[0120] Each electronic processing unit (1121a, 1121b, 1121c, 1121d) can use other activation functions, such as the RELU function, in addition to the sigmoid function.

[0121] In addition to performing function operations, the electronic processing units (1121a, 1121b, 1121c, 1121d) can store result values ​​or external input values ​​for use in calculations. This memory function enables various calculations to be performed by comparing calculated values ​​in the next sequence with stored values ​​or by using stored values ​​for function calculations.

[0122] The electronic processing unit has a communication function and can exchange data with surrounding electronic processing units or external processors through connected communication lines.

[0123] The second light source (1122a, 1122b, 1122c, 1122d) corresponds to ai(l) of the neural network and is used as an input value for the next layer. Light rays in other directions originating from the first light source 1 (101a, 1101b) are also collected by other light detectors located on the second substrate (1130) and connected as input values ​​for the next layer through an electronic processing unit.

[0124] The electronic processing unit (1121a, 1121b, 1121c, 1121d) is an electronic integrated circuit formed on a semiconductor substrate and can be implemented through an analog circuit or a digital circuit.

[0125] The lenses used in this embodiment can be implemented using refractive lenses or diffractive optical elements.

[0126] The first light source (1101a, 1101b) and the first lens (1102a, 1102b) corresponding thereto can be formed to be spaced apart by the focal length of the first lens (1102a, 1102b) within an error of 20%.

[0127] In addition, the spatial light modulators (1103a, 1103b) and the second lenses (1104a, 1104b) corresponding thereto, respectively, may be formed to be spaced apart from each other by the focal length of the second lenses (1104a, 1104b) within an error of 20%, and each photodetector (1120a, 1120b, 1120c, 1120d) may be formed to be spaced apart from the third lens (1105) by the focal length of the third lens (1105) within an error of 40%.

[0128] In particular, when the spatial light modulator (1103a, 1103b) is spaced apart from the second lens (1104a, 1104b) by the focal length of the second lens (1104a, 1104b), and the lens (1105) is spaced apart from the photodetector (1120) by the focal length of the third lens (1105), the image of the spatial light modulator (1103a, 1103b) is generated on the photodetector, so that cross-talk can be reduced.

[0129] However, even if the distance between the spatial light modulator (1103a, 1103b) and the second lens (1104a, 1104b) is not exactly the focal length but slightly off, the distance between the third lens (1105) and the photodetector (1120a, 1120b, 1120c, 1120d) can be adjusted so that the image of the pixel of the spatial light modulator (1103a, 1103b) is focused on the photodetector (1120a, 1120b, 1120c, 1120d), so that a certain degree of tolerance can be achieved.

[0130] Likewise, the distance between the first light source (1101a, 1101b) and the corresponding first lens (1102a, 1102b) can be adjusted to maintain performance even if the distance between other lenses or elements deviates slightly from the focal length.

[0131] For example, a plurality of first lenses (1102a, 1102b) corresponding to a plurality of light sources can be formed on a single substrate, a plurality of spatial light modulators (1103a, 1103b) can also be formed on a single substrate, and a plurality of second lenses (1104a, 1104b) can also be formed on a single substrate. This structure can facilitate the volume and optical alignment of the entire system.

[0132] Fig. 8 is an example of applying the optoelectronic module implemented in Fig. 6 in cascade.

[0133] Referring to FIG. 8, the first optical computation layer (1201) is repeated and appears in the second optical computation layer (1202), and an arbitrary number of neural network layers can be implemented. The output values ​​from the preceding layer can be connected to the inputs of the subsequent layer.

[0134] Figure 9 is a conceptual diagram of an optoelectronic module using a spatial light modulator according to a second embodiment of the present invention.

[0135] The optoelectronic module (1030) using the spatial light modulator according to the present embodiment can be constructed in substantially the same configuration as the optoelectronic module (1010) of FIG. 6, except for the configuration of the photodetector. Therefore, a repeated description of the same components as the optoelectronic module (1010) of FIG. 6 is omitted.

[0136] Referring to FIG. 9, an optical optoelectronic module (1030) according to a second embodiment of the present invention includes a plurality of first photodetectors (1321a, 1321b) and a plurality of second photodetectors (1323a, 1323b). In addition, it may further include a plurality of electronic processing units (1322a, 1322b) and corresponding second light sources (1324a, 1324b).

[0137] The output values ​​of the first photodetector (1321a) and the second photodetector (1323a) can be connected to an electronic processing unit (1322a), and the output values ​​of the second photodetector (1321b) and the second photodetector (1323b) can be connected to an electronic processing unit (1322b).

[0138] The structure of this embodiment offers the advantage of being able to perform various operations using the values ​​of two photodetectors. The easiest example is subtraction. For example, Let's assume that we are calculating .

[0139] Subtraction using light is possible using principles like destructive interference, but this increases the complexity of optical computing circuits, as it requires adjusting for path differences. Subtraction is not easy when two light beams enter a photodetector that only detects light intensity.

[0140] Therefore, in this embodiment, the light corresponding to the (+) term is collected by the second photodetector (1323a, 1323b), and the light corresponding to the (-) term is collected by the first photodetector (1321a, 1321b), and the two results are subtracted to calculate.

[0141] Using the expression in Figure 7 , , , If we substitute and calculate, the outputs of the two detectors (1323b, 1321b) are , , and the result of subtraction in the electronic processing unit (1322b) is as follows: Mathematical Expression 3.

[0142] [Equation 3]

[0143]

[0144] The electronic processing unit (1322b) calculates the bias and sigmoid function together with the subtraction. can be obtained.

[0145] Even if there are an arbitrary number of terms as in mathematical expression 1, if the weights of the rays corresponding to the (+) term are set to the original model value and the weights of the rays corresponding to the (-) term are set to 0, only the rays corresponding to the (+) term can be collected and added in the photodetector (1323b).

[0146] Similarly, the rays corresponding to the (-) term can be collected and sent to the photodetector (1321b) for summing. Thus, general neural network-related operations involving negative weights can be performed in this manner.

[0147] Each electronic processing unit (1322a, 1322b) can use the output of not only two surrounding photodetectors but also a plurality of adjacent photodetectors as input, and can implement various functions other than subtraction through electronic circuits.

[0148] In practice, subtraction and sigmoid function operations can be implemented using analog or digital electronic circuits. While this embodiment illustrates connections between multiple photodetectors and electronic processing units, connections between multiple electronic processing units can also achieve the same effect.

[0149] Each electronic processing unit (1322a, 1322b) can also store a bias value, amplify the current input from the photodetector, or convert the result of the operation into a current value proportional to the result when sending it to the input of the second light source.

[0150] Fig. 10 shows a cascaded configuration of the optoelectronic modules implemented in Fig. 9, which can implement an arbitrary number of neural network layers (1401, 1402).

[0151] Fig. 11 is a conceptual diagram of an optoelectronic module using a spatial light modulator according to a third embodiment of the present invention.

[0152] The optoelectronic module (1050) using a spatial light modulator according to the present embodiment uses a single-stage lens (1504a, 1504b, 1504c), unlike the two-stage lens used in the next stage after the spatial light modulator in the optoelectronic module (1010) of FIG. 6 and the optoelectronic module (1030) of FIG. 9. In the present embodiment, the single-stage lens (1504a, 1504b, 1504c) is referred to as the fourth lens (1504a, 1504b, 1504c).

[0153] The fourth lens (1504a, 1504b, 1504c) sends the light beams passing through the pixels of the corresponding spatial light modulators (1503a, 1503b, 1503c) in different directions, and forms small spots on the surface of the photodetectors (1521a, 1521b, 1521c) on the second substrate (1520).

[0154] Additionally, the fourth lens (1504a, 1504b, 1504c) serves to collect some of the rays that are irradiated from the plurality of first light sources (1501a, 1501b, 1501c) and pass through pixels with different relative positions onto one detector.

[0155] At this time, the fourth lens (1504a, 1504b, 1504c) can be formed at a position where the image of the pixel in the corresponding spatial light modulator (1503a, 1503b, 1503c) is focused on the surface of the photodetector (1521a, 1521b, 1521c).

[0156] In this embodiment, each of the first light sources (1501a, 1501b, 1501c) and the first lenses (1502a, 1502b, 1502c) corresponding thereto can be formed to be spaced apart by the focal length of the first lenses (1502a, 1502b, 1502c) within an error of 20%.

[0157] Accordingly, each fourth lens (1504a, 1504b, 1504c) can cause the image of the pixel of the spatial light modulator (1503a, 1503b, 1503c) corresponding to each fourth lens (1504a, 1504b, 1504c) to be focused on the surface of the photodetector (1521a, 1521b, 1521c) within an error of 20%.

[0158] For example, the fourth lens (1504a, 1504b, 1504c) can be implemented using a lens combining a prism and a spherical lens or a diffractive optical element.

[0159] In the case of Fig. 11, the light detectors (1521a, 1521b, 1521c) have a structure in which the light from the three first light sources (1501a, 1501b, 1501c) is multiplied by weights in the spatial light modulators (1503a, 1503b, 1503c) and then added in the light detectors (1521a, 1521b, 1521c).

[0160] The photocurrent obtained in this manner calculates the bias and sigmoid function in the electronic processing unit (1522a, 1522b, 1522c), and is connected to the electronic processing unit (1522a, 1522b, 1522c) in the vicinity to perform various operations, and then sends the output value to the next layer light source (1523a, 1523b, 1523c).

[0161] The present invention provides an optical computer structure and technology that increases the calculation speed by performing approximately NXM multiplications and additions at once without complicating the physical structure of the connection lines or generating noise as in an electric circuit even when the number of input and output terminals increases to N and M, respectively, by implementing an optical computer structure using optical connections instead of purely electronic circuits.

[0162] When the pixel values ​​of the spatial light modulator are already determined, parallel computation is possible because the optical paths from the light source to the photodetector overlap, but photons on different paths do not interfere with each other, and the computation time is very short because photons arrive at the photodetector at the speed of light.

[0163] Furthermore, the time delay in computation occurs in the electronic processing unit, but since it is performed in parallel across all outputs, it can be viewed as equivalent to one instruction cycle of a digital electronic circuit. When configuring L multilayer neural networks, the actual computation speed increases by NXMXL times because computation is performed simultaneously across all layers.

[0164] In the drawings 6 to 11 described above, an optoelectronic module using a spatial light modulator and an optical neural network computer including the same are described.

[0165] However, these optical neural network computers utilize slow-switching spatial light modulators, which can be time-consuming to update when performing calculations on different sets of weights for the same input data. Furthermore, the need for weight set updates and the slow switching speed of the spatial light modulator can hinder the parallel processing speed of the optical neural network computer when performing algorithms that require intermediate weight changes rather than using fixed weight values.

[0166] Therefore, below, an optoelectronic module using a smart pixel optical modulator and an optical neural network computer including the same will be described through FIGS. 12 to 15.

[0167] FIG. 12 is a conceptual diagram of an optical neural network computer using a smart pixel optical modulator according to the first embodiment of the present invention.

[0168] The illustrated optoelectronic module (1060) includes a plurality of first light sources (1601a, 1601b), a plurality of first lenses (1602a, 1602b), a plurality of smart pixel light modulators (SPLMs) (1603a, 1603b), a plurality of photodiodes (PDs) (1620a, 1620b, 1620c, 1620d), and a lens unit (1604a, 1604b, 1605).

[0169] A plurality of first light sources (1601a, 1601b) may be formed at regular intervals on the first substrate (1600), but may also be formed on different substrates.

[0170] The optoelectronic module (1060) according to the first embodiment of the present invention may further include a plurality of electronic processing (EP) units (1621a, 1621b, 1621c, 1621d) and a plurality of second light sources (1622a, 1622b, 1622c, 1622d).

[0171] A plurality of photodetectors (1620a, 1620b, 1620c, 1620d), a plurality of electronic processing units (1621a, 1621b, 1621c, 1621d), and a plurality of second light sources (1622a, 1622b, 1622c, 1622d) may be formed on a second substrate (1630), but may alternatively be formed separately on another substrate, or one or more elements may be formed on the same substrate. For example, the first substrate (1600) and the second substrate (1630) may be semiconductor substrates or PCBs.

[0172] In FIG. 12, the first light source (1601a, 1601b), the first lens (1602a, 1602b), the smart pixel light modulator (1603a, 1603b), etc. are expressed as two, and the light detector (1620a, 1620b, 1620c, 1620d), the electronic processing unit (1621a, 1621b, 1621c, 1621d), and the second light source (1622a, 1622b, 1622c, 1622d) are expressed as four, but these are only examples, and the number of each element can be changed and designed as needed.

[0173] For example, in the present invention, the light source may be an LED or a semiconductor laser, particularly a vertical surface-emitting laser. The intensity of light can be adjusted by varying the intensity of the current flowing through the light source, thereby individually controlling the input values.

[0174] Light emitted from a plurality of first light sources (1601a, 1601b) passes through a plurality of corresponding first lenses (1602a, 1602b), and then passes through a corresponding smart pixel light modulator (1603a, 1603b). The first lenses (1602a, 1602b) reduce the divergence angle of the light emitted from the corresponding first light sources (1601a, 1601b), and preferably modulate the light into parallel light.

[0175] In Fig. 12, four representative light rays are shown among the light emitted from each first light source (1601a, 1601b), and the four light rays pass through individual pixels in the smart pixel light modulator (1603a, 1603b).

[0176] The smart pixel light modulators (1603a, 1603b) are formed to correspond to each of the plurality of first lenses (1602a, 1602b), and adjust the intensity of light passing through the corresponding first lenses (1602a, 1602b) through each pixel having a weight stored in a memory.

[0177] In addition, the smart pixel light modulator (1603a, 1603b) is configured in a form in which a plurality of pixels are arranged, and a light detector, an electronic processing unit (1606a, 1606b), and a light source (1607a, 1607b) are assigned to each pixel. Here, the light detector (1606a, 1606b) converts light passing through the first lens (1602a, 1602b) into a current signal, and the electronic processing unit (1606a, 1606b) amplifies the current signal or converts the result of calculating the current signal by multiplying or adding the current signal with a weight value stored in a memory into a current signal and outputs the result to the connected light source (1607a, 1607b).

[0178] In this way, the smart pixel light modulators (1603a, 1603b) do not compromise the efficiency of parallel computation because each pixel is assigned a photodetector, an electronic processing unit, and a light source, so they do not need to be connected to surrounding pixels. Most importantly, because these electronic components have a fast switching speed, weight updates are possible at a speed of several hundred MHz, which is much faster than the speed of several kHz of the spatial light modulator (LCD, DMD) described above through FIGS. 6 to 11.

[0179] Each pixel's memory stores the weights or bias values ​​to be applied according to the program. Ultimately, a smart pixel light modulator can be thought of as an array of optical repeaters, each with its own memory.

[0180] Meanwhile, light rays passing through the smart pixel light modulators (1603a, 1603b) are input to the lens unit (1604a, 1604b, 1605). The lens unit (1604a, 1604b, 1605) is formed between a plurality of smart pixel light modulators (1603a, 1603b) and a plurality of photodetectors (1620a, 1620b, 1620c, 1620d).

[0181] Additionally, the lens units (1604a, 1604b, 1605) serve to focus light emitted from different first light sources (1601a, 1601b) passing through pixels at the same relative positions of each smart pixel light modulator (1603a, 1603b) onto one light detector (1620a, 1620b, 1620c, 1620d).

[0182] In an embodiment of the present invention, the lens units (1604a, 1604b, 1605) are formed corresponding to the plurality of smart pixel light modulators (1603a, 1603b), respectively, and include a plurality of second lenses (1604a, 1604b) that focus light rays to a single point, and a third lens (1605) that passes all light rays that have passed through the plurality of second lenses (1604a, 1604b). If the light rays incident on the second lenses (1604a, 1604b) are parallel light rays, they are focused at a focus.

[0183] The light passing through each smart pixel light modulator (1603a, 1603b) passes through the corresponding second lens (1604a, 1604b), is focused, and then spreads out again, where it passes through the third lens (1605) again.

[0184] When a ray passes through the second lens (1604a, 1604b), the angle of the ray changes depending on the height of the ray away from the optical axis of the second lens (1604a, 1604b). The ray (1610a) passing through the highest position on the optical axis of the second lens (1604a) and the ray (1610b) passing through the highest position on the optical axis of the second lens (1604b) have the same angle with respect to the horizontal axis.

[0185] The two light rays (1610a, 1610b) enter the third lens (1605) while forming parallel light, and thus converge at the same point on the focal plane of the third lens (1605). In Fig. 12, the two light rays converge on one photodetector (1620d).

[0186] Similarly, light rays (1611a) passing through the second lens (1604a) and light rays (1611b) passing through the second lens (1604b), which are rays at relatively the same position, pass through the third lens (1605) and are then collected on one photodetector (1620c).

[0187] In addition, light rays (1612a) passing through the second lens (1604a) and light rays (1612b) passing through the second lens (1604b) pass through the third lens (1605) and are then collected on one photodetector (1620b), and light rays (1613a) passing through the second lens (1604a) and light rays (1613b) passing through the second lens (1604b) pass through the third lens (1605) and are then collected on one photodetector (1620a).

[0188] The photodetectors (1620a, 1620b, 1620c, 1620d) can usually be composed of optical semiconductor devices such as photodiodes. The current flowing through the photodetectors (1620a, 1620b, 1620c, 1620d) is ultimately the sum of the intensities of light arriving at each photodetector (1620a, 1620b, 1620c, 1620d). , It has a value proportional to .

[0189] FIG. 13 is a conceptual diagram of an optical neural network computer using a smart pixel optical modulator including a fan-out according to a second embodiment of the present invention, and FIG. 14 is a conceptual diagram of the smart pixel optical modulator of FIG. 13.

[0190] The illustrated optoelectronic module (1070) includes a plurality of electrical inputs (1700), a plurality of smart pixel optical modulators (1701), lens units (1702, 1703), and a plurality of photodiodes (PDs) (1704). The optoelectronic module (1070) is a further simplified structure of the structure of FIG. 12, in which the first lens is eliminated and instead an electrical fan-out structure (1707) is included in the smart pixel optical modulator (1701).

[0191] The structure of Fig. 13 can perform neural network operations with weights having (-) signs, as shown in Fig. 9. In Fig. 9, among the weights, light rays passing through the spatial light modulator pixels corresponding to weights having (+) terms and weights having (-) terms are collected in separate photodetectors and an operation is performed through subtraction in an electronic processing unit.

[0192] However, in the structure of FIG. 13, since the weight update speed of the smart pixel optical modulator (1701) is fast, the result of the calculation using the weight corresponding to the (+) term is first stored in the memory of the electronic processing unit (1705), and then updated with the weight corresponding to the (-) term and collected in the photodetector (1704) for the same input value, and the electronic processing unit (1705) performs the calculation by subtracting the (-) term calculation result from the (+) term calculation result stored in the memory, thereby obtaining the same result as that shown in FIG. 9.

[0193] This further simplifies the hardware configuration, as it eliminates the need for two photodetectors and two spatial light modulator pixels, each corresponding to two optical channels, for each output node. Compared to conventional spatial light modulators, the smart pixel light modulator (1701) is thus highly effective in applying algorithms that require updating weights mid-calculation for the same input values.

[0194] A more detailed structure of the smart pixel optical modulator (1701) is shown in Fig. 14. The electrical fan-out structure is a structure in which the electrical input signal is branched into multiple branches through wires, and is simpler than a structure in which the electrical input signal is optically connected through a lens. Therefore, by applying the electrical fan-out structure (1707) to the smart pixel optical modulator (1701) as shown in Figs. 13 and 14, the overall hardware structure of the optoelectronic module can be further simplified.

[0195] The smart pixel light modulator (1701) is configured in a form in which a plurality of pixels are arranged, and each pixel is assigned an electronic processing unit (1708) and a light source (1709).

[0196] An electrical input signal is distributed to each of the pixels through a fan-out structure (1707). That is, the input signal is connected to an electronic processing unit (1708) through the fan-out structure (1707), and the electronic processing unit (1708) converts the result of calculating the input signal with a weight value stored in a memory into a current signal and outputs the result to a light source (1709). Here, the calculation may be, for example, a multiplication operation, and therefore, the intensity of the output light source of each pixel has a characteristic that is proportional to the product of the input signal and the weight value.

[0197] FIG. 15 is a conceptual diagram of an optical neural network computer using a smart pixel optical modulator according to a third embodiment of the present invention.

[0198] FIG. 15 is a structure in which the distance between the second lens (1702) and the third lens (1703) is reduced and they are positioned closer together than in the structure described in FIG. 13. In FIG. 13, the second lens (1702) and the third lens (1703) form a relay optical system, so that parallel light passes between the two lenses. Therefore, even if the distance between the two lenses is reduced as in FIG. 15, the same result is achieved in which the image of the smart pixel optical modulator (1801) is focused on the first photodetector (1804).

[0199] Additionally, the second lens (1802) and the third lens (1803) shown in Fig. 15 can be combined into a single optical component to integrate their functions, or they can be combined into a single lens using a diffractive optical element (DOE) to integrate their functions. In this case, the hardware structure of the optoelectronic module can be further simplified.

[0200] The concept of an optoelectronic module (2010) using a spatial light modulator (2400) will be explained through FIGS. 16 to 19.

[0201] Fig. 16 is a conceptual diagram of an optoelectronic module (2010) using a spatial light modulator (2400) according to the fourth embodiment of the present invention.

[0202] An optoelectronic module (2010) using a spatial light modulator (2400) according to the present invention may include a substrate (2100), a first light source (2200), a lens unit (2300), a spatial light modulator (2400), a photodetector (2500), and an electronic processing unit (2600).

[0203] The substrate (2100) may be formed with a plurality of first light sources (2200), a plurality of light detectors (2500), and a plurality of electronic processing units (2600).

[0204] This means that one or more elements may be formed on the same substrate, but alternatively, they may be formed separately on different substrates.

[0205] Additionally, the substrate (2100) may be formed as a semiconductor substrate or a PCB (Printed Circuit Board).

[0206] Meanwhile, the light source (2200) may be a light emitting diode (LED), a semiconductor laser, or a vertical-cavity surface-emitting laser.

[0207] The light source (2200) can control the amount of light by controlling the applied current, and the input values ​​of the current flowing to multiple light sources can be individually controlled.

[0208] Light from a light source (2200) can pass through a lens unit (2300) corresponding to the light source and then pass through a corresponding spatial light modulator (2400).

[0209] The lens unit (2300) is formed between a plurality of spatial light modulators (2400) and a plurality of light detectors (2500), and can implement a function of calculating a convolution of the light source array and the pixel array of the spatial light modulator (2400) by using the function of gathering light emitted from different first light sources (2200) passing through pixels at different relative positions of each spatial light modulator (2400) into one light detector (2500).

[0210] Additionally, the lens unit (2300) may include a first lens (2310), a second lens (2320), and a third lens (2330).

[0211] The first lens (2310) is formed to correspond to each of the plurality of light sources (2200) and can reduce the divergence angle of light emitted from the corresponding light source (2200).

[0212] The second lens (2320) may be provided in multiple units at positions corresponding to each of the multiple spatial light modulators (2400) to focus the light beams into one point.

[0213] The third lens (2330) may be provided in multiple numbers to correspond to each second lens (2320) by passing the light rays that have passed through the second lens (2320).

[0214] A light ray passing through a pixel of the same spatial light modulator (2400) after passing through the first lens (2310) may have the same angle after passing through the second lens (2320).

[0215] Additionally, the lens unit (2300) can be implemented using a refractive lens, a diffractive optical element, a meta lens, etc.

[0216] In addition, the light ray passing through the spatial light modulator (2400) can pass through the corresponding second lens (2320), be focused, and then spread out again to pass through the third lens (2330). More specifically, when passing through the second lens (2320), the angle of the light ray varies depending on the height of the light ray away from the optical axis of the second lens (2320), and the first light ray passing through the highest position on the optical axis of the second lens (2320) and the second light ray passing through the highest position on the optical axis of the second lens (2320) can have the same angle with respect to the horizontal axis.

[0217] Accordingly, the first light source (2200) and the first lens (2310) corresponding thereto may be formed to be spaced apart by the focal length of the first lens (2310), and the separation distance may include a certain error that deviates from the focal length. In addition, each spatial light modulator (2400) and the second lens (2320) corresponding to each spatial light modulator may be formed to be spaced apart by a first distance, and each photodetector (2500) may be formed to be spaced apart by a second distance from the third lens. Here, the second distance may satisfy a relationship in which the distance at which the image of the pixel of the spatial light modulator (2400) is formed on the photodetector (2500) does not deviate by more than a certain error value.

[0218] The spatial light modulator (2400) is formed in a plurality of units corresponding to each of the plurality of first lenses (2310), and can control the intensity of light passing through the first lens (2310) through each pixel having a preset weight.

[0219] Additionally, the relative position of the spatial light modulator (2400) may mean the coordinates or position of the spatial light modulator pixel on the spatial light modulator plane based on the central axis of the first lens (2310), second lens (2320), or third lens (2330).

[0220] The spatial light modulator (2400) has spatial light modulator pixel arrays with sizes corresponding to the lens diameter, and the relative positions of the pixels can be determined based on the corresponding lens central axis. For example, pixels located on the central axis of each lens can be said to be at the same relative position.

[0221] The spatial light modulator (2400) controls the intensity of incoming light to pass through at different transmittances according to voltage, so it can be used as a liquid crystal display panel (LCD) that can control the intensity of each light beam to a predetermined value.

[0222] Meanwhile, even if the light originates from the same light source (2200), the intensity of the light passing through the pixels may vary depending on the transmittance of each pixel in the LCD panel of the spatial light modulator (2400).

[0223] For example, if the illuminance of the light source (2200) is 1.0 W / ㎡, if the transmittance of the LCD pixel of the spatial light modulator (2400) is 0.10, the illuminance of the output light may be 0.10 W / ㎡, and if the transmittance is 0.05, the illuminance of the output light may be 0.05 W / ㎡.

[0224] In addition, the transmittance of the LCD panel of the spatial light modulator (2400) can be adjusted in more than 256 steps depending on the voltage, and the product of the weights and input values ​​in the artificial neural network can be implemented.

[0225] Here, the input value is the intensity of the light source (2200), the transmittance of the spatial light modulator (2400) corresponds to the weight, the transmittance of the spatial light modulator (2400) corresponds to the weight, and the number of pixels can be adjusted to be equal to the number of elements of the kernel.

[0226] The photodetector (2500) may be composed of an optical semiconductor device such as a photodiode.

[0227] The photodetector (2500) is formed spaced apart from a plurality of spatial light modulators (2400) and can obtain a current value according to the intensity of light.

[0228] At this time, the photodetector (2500) may be provided in multiple numbers corresponding to the third lens (2330) or positioned within the aperture of the third lens (2330), and each photodetector (2500) may perform different convolution operations on light passing through the pixels of the spatial light modulator (2400) at different relative positions.

[0229] At this time, the current flowing to the photodetector (2500) is the sum of the intensities of light arriving at each photodetector. , can have a value proportional to .

[0230] Additionally, the photodetector (2500) can be arranged in multiple numbers in the optoelectronic module (2010) by adjusting the ratio of the focal length f2 of the second lens (2320) and the focal length f3 of the third lens (2330) within the diameter of the lens corresponding to one input node.

[0231] For example, the number of photodetectors (2500) is N p 2 When given as, the lens part (2300) is placed within the diameter D, and the sub-array of the spatial light modulator (2400) is (N p × N m ) 2 is given by D = N p × N m × d, so f3 / f2= N m If satisfies , N is within the area corresponding to the lens. p × N p A pixel image of the array is formed, and a photodetector (2500) can be installed thereon.

[0232] This is applicable when the array of input nodes is N × N by implementing an optical convolutional neural network structure using optical connections instead of pure electronic circuits, and the size of the convolution kernel array (N m × Nm ) has some limitations, but the size of the array of input / output nodes is theoretically unlimited or can be very large in practice, so the amount of parallel data that can be optically processed in one instruction cycle is N. 2 × N m 2 When using L layers of convolution, the parallel processing capacity is N 2 × N m 2 × It has the advantage of being able to overcome the performance limits of conventional electronic parallel computers by increasing L.

[0233] Also, N=10000, N m = 165, L = 20, the parallel processing capacity is 5.4 × 10 13 With MAC (multiply accumulate operation) / cycle, even if the number of input terminals increases, the physical structure of the connection line does not become complicated or noise is generated like an electric circuit, and approximately N 2 × N m 2 × It can provide the effect of speeding up calculations by performing L multiplications and additions at once.

[0234] In addition, since the light rays arriving at the photodetector (2500) do not interfere with photons of different paths even though the optical paths from the light source to the photodetector (2500) overlap when the pixel values ​​of the spatial light modulator (2400) are already determined, parallel calculation is possible, so the calculation time can be shortened.

[0235] The electronic processing unit (2600) is an electronic direct circuit formed on a semiconductor substrate and can be implemented through an analog circuit or a digital circuit.

[0236] The electronic processing unit (2600) adds the current value or its amplified value obtained from each corresponding photodetector (2500) and the bias value stored in each electronic processing unit (2600), calculates a sigmoid function α using the value as input, and then obtains a neural network output value.

[0237] At this time, each output value of the electronic processing unit (2600) can be transmitted to another connected light source (not shown).

[0238] Therefore, the output values ​​of the electronic processing unit (2600) are , can be adjusted proportionally.

[0239] In addition, the electronic processing unit (2600) can use other activation functions such as the RELU function in addition to the sigmoid function, and in addition to the function operation, it can store the result value or external input value and use it for the operation, so that the memory function of the electronic processing unit (2600) can compare the calculated value in the next sequence with the stored value or perform various calculations using the stored value.

[0240] Additionally, the electronic processing unit (2600) can exchange data with a surrounding electronic processing unit (2600) or an external processor through a locally connected communication line, including a communication function.

[0241] These electronic processing units (2600) can electronically perform functions such as pooling and local response normalization (LRN) required in CNN through communication and calculation with electronic processing units (2600) existing in the vicinity.

[0242] Additionally, the electronic processing unit (2600) can store a bias value, and can also perform a role of amplifying the current input from the photodetector (2500) or converting the result of the operation into a current value proportional to the result when sending the result to the input of the second light source.

[0243] Additionally, the second light source (not shown) located on the substrate (2100) of Fig. 16 is a neural network. , and can be used as input values ​​for the next layer.

[0244] Light rays in other directions starting from the first light source (2200) can also be collected at other light detectors located on the second substrate (2800) and connected as input values ​​of the next layer through the electronic processing unit (2600) located on the second substrate (2800).

[0245] In addition, the spatial light modulator (2400) and the corresponding second lens (2320) may be formed to be spaced apart by the focal length of the second lens (2320) within a certain error, and each light detector (2500) may be formed to be spaced apart from the third lens (2330) within a certain error.

[0246] At this time, the spatial light modulator (2400) and the light detector (2500) can satisfy the positional relationship between the object and the image within a certain error by means of the imaging optical system composed of the second lens and the third lens.

[0247] Here, when the spatial light modulator (2400) is spaced apart from the second lens (2320) by the focal length of the second lens (2320) and the third lens (2330) is spaced apart from the photodetector (2500) by the focal length of the third lens (2330), the image of the pixel of the spatial light modulator (2400) is generated on the photodetector (2500), so that cross-talk that may occur between neighboring photodetectors (2500) can be reduced.

[0248] Meanwhile, even if the distance between the spatial light modulator (2400) and the second lens (2320) is not exactly the focal length but slightly off, a tolerance can be allowed so that the image of the pixel of the spatial light modulator (2400) is formed on the photodetector (2500) by adjusting the distance between the third lens (2330) and the photodetector (2500).

[0249] Additionally, the distance between the first light source (2200) and the corresponding first lens (2310) can be adjusted to maintain performance even when the distance between other lenses or elements deviates from the focal length.

[0250] For example, a plurality of first lenses (2310) corresponding to a plurality of light sources may be formed on another substrate, a plurality of spatial light modulators (2400) may also be formed on another substrate, and a plurality of second lenses (2320) and third lenses (2330) may also be formed on another substrate.

[0251] This structure can have the effect of facilitating the volume or optical alignment of the entire system.

[0252] Below, a three-dimensional example of an optoelectronic module (2010) using a spatial light modulator (2400) and the pixel spacing of the spatial light modulator (2400) will be described through FIGS. 17 and 18.

[0253] Fig. 17 is a three-dimensional example of an optoelectronic module (2010) using the spatial light modulator (2400) of Fig. 16, and Fig. 18 is an example of the spacing of pixels of the spatial light modulator (2400) of Fig. 17.

[0254] f1, f2, and f3 in FIG. 17 may be focal lengths of the first lens (2310), the second lens (2320), and the third lens (2330).

[0255] The spatial light modulator (2400) pixels of FIGS. 17 and 18 are composed of blocks in a 3x3 array, and the spacing between blocks is indicated by a.

[0256] The spacing a is equal to the spacing or diameter between the lenses of the lens unit (2300) of Fig. 16, and may be equal to the spacing of the input nodes (first light source) or output nodes (photodetectors), so the size of the block array may be equal to the size of the input node array.

[0257] Here, inside the block, it is again composed of a 3Х3 array in the form of a sub-array, where the size and spacing of the pixels are respectively and can be expressed as d.

[0258] These sub-arrays correspond to kernels or filter masks in convolution operations, and implement weight values ​​through the transmittance of the spatial light modulator (2400), and when performing convolution based on the relative position of the pixel from the center of the sub-array, it is possible to determine which output node in the vicinity the light passing through the pixel is directed to after passing through the second lens (2320) and the third lens (2330).

[0259] Also, since the size of the sub-array is the same as the array size of one input node and the output node where convolution is performed, if the size of the sub-array is 5Х5, it can be connected to the output nodes of the surrounding 5Х5 arrays to perform a convolution operation.

[0260] Meanwhile, if all pixels are spaced apart by a distance d, they are positioned to satisfy a=5d, and since the second lens (2320) and the third lens (2330) form a relay optical system and focus the image of the pixels on the surface of the photodetector (2500), the focal length f3 of the third lens (2330) can be five times the focal length f2 of the second lens (2320).

[0261] In general, if the array size of the convolution kernel is Nm X Nm, , and the aperture of the lens is And It could be.

[0262] Here is a fixed value as the f-number of the second lens (2320).

[0263] Also, the f / # of the third lens (2330) is Therefore, the spot diameter due to the diffraction limit of the third lens is ㎛, and both the spot diameter and lens diameter due to the diffraction limit are N m Because it is proportional to N m Even if the diffraction limit is increased, there may be no noise due to the diffraction limit.

[0264] For example, if the cell spacing is 20 μm and N m If this is 5, then D = 100㎛ When it is 2, can be 10 days.

[0265] Therefore, when the diffraction limit spot diameter of the third lens (2330) is 10㎛, since D is 100㎛, there may be no crosstalk between neighboring photodetectors (2500).

[0266] On the other hand, the geometric optical aberration becomes important between the pixels of the second lens (2320) and the spatial light modulator (2400), and in the case of a system with f / # of 2, if three lenses are used, the geometric optical angle aberration is 3 m㎭, so when f2 is 3.3 mm, the geometric optical spot diameter is 10 ㎛, so it may exceed 50% of the input node spacing, causing serious crosstalk.

[0267] In addition, the geometric optical limit of the array of kernels is about 165 X 165, but kernels usually do not require a large array size and do not place restrictions on the size of input or output arrays, so the parallel processing capacity of the optoelectronic module (2010) can be greatly increased.

[0268] Fig. 19 is a multi-output conceptual diagram of an optoelectronic module (2010) using the spatial light modulator (2400) of Fig. 16.

[0269] The optoelectronic module (2010) according to the present invention can be applied in cascade, and can further include a plurality of first photodetectors (2500, 2521, 2522, 2523) and a plurality of second photodetectors (2511, 2512, 2513, 2514) whose output values ​​from the preceding layer are connected to the inputs of the subsequent layer, a plurality of first electron processing (2600, 2621, 2622, 2623) and a plurality of second electron processing units (2611, 2612, 2613, 2614), and a third light source (2721, 2723, 2725, 2727) and a fourth light source (2722, 2724, 2726, 2728) corresponding to the plurality of second electron processing units (2611, 2612, 2613, 2614).

[0270] For example, the output values ​​of the first photodetector (2521) and the second photodetector (2512) are respectively connected to the first electronic processing unit (2621) and the second electronic processing unit (2612), and the first electronic processing unit (2621) and the second electronic processing unit (2612) can be connected.

[0271] Additionally, the output values ​​of the first photodetector (2521) and the second photodetector (2512) can be connected together to either the first electronic processing unit (2621) or the second electronic processing unit (2612).

[0272] The structure of this embodiment provides the advantage of being able to perform various operations using the values ​​of two photodetectors, so that light corresponding to the positive (+) term of the weight can be collected in the second photodetector (2511, 2512, 2513, 2514), light corresponding to the negative (-) term can be collected in the first photodetector (2500, 2521, 2522, 2523), and these two results can be subtracted to perform calculations.

[0273] for example, class is positive Assuming this is negative, , , , , , The spatial light modulator (2400) weights can be set as follows, and the outputs of the two detectors are each and It could be.

[0274] The result of subtraction in the second electronic processing unit (2612) is as shown in mathematical expression 4.

[0275] [Equation 4]

[0276]

[0277] Additionally, the second electronic processing unit (2612) calculates the bias and sigmoid function together with the subtraction. can be obtained.

[0278] In this way, the light rays corresponding to the (+) term have their weights set to the original model values, the light rays corresponding to the (-) term have their weights set to 0, and then only the light rays corresponding to the (+) term can be collected and added in the second photodetector (2512).

[0279] Additionally, the rays corresponding to the (-) term can be collected and sent to the first photodetector (2521) to be summed, thereby performing general neural network-related operations including negative weights.

[0280] At this time, each electronic processing unit (2600, 2621, 2622, 2623, 2611, 2612, 2613, 2614) can use the output of a plurality of adjacent photodetectors (2500, 2521, 2522, 2523, 2511, 2512, 2513, 2514) as input, and can implement various functions other than subtraction through electronic circuits.

[0281] The components according to the present invention are defined by functional distinctions rather than physical distinctions, and can be defined by the functions each component performs. Each component may be implemented as hardware or as program code and processing units that perform each function, and the functions of two or more components may be implemented by including them in a single component. Therefore, it should be noted that the names given to the components in the following embodiments are not intended to physically distinguish each component, but rather to suggest the representative functions performed by each component, and that the technical spirit of the present invention is not limited by the names of the components.

[0282] An optoelectronic module using a spatial light modulator according to the fourth embodiment of the present invention described above is included in an optical convolutional neural network computer.

[0283] In the drawings 16 to 19 described above, an optoelectronic module using a spatial light modulator and an optical convolutional neural network computer including the same are described.

[0284] However, these optical neural network computers utilize slow-switching spatial light modulators, which can be time-consuming to update weight sets when performing calculations on different sets of weights for the same input data. Furthermore, the need to update these weight sets and the slow switching speed of the spatial light modulator can hinder the parallel processing speed of optical neural network computers.

[0285] Therefore, below, an optoelectronic module using a smart pixel optical modulator and an optical convolutional neural network computer including the same will be described through FIGS. 20 to 26.

[0286] FIG. 20 is a conceptual diagram of an optoelectronic module using a smart pixel optical modulator according to a fifth embodiment of the present invention.

[0287] An optoelectronic module (2060) using a smart pixel light modulator (2063) according to the present invention may include a first light source (2061), a first lens (2062), a smart pixel light modulator (SPLM) (2063), a second lens (2064), a third lens (2065), a photodiode (PD) (2066), an electronic processing unit (2067), and a second light source (2068).

[0288] The illustrated optoelectronic module (2060) is implemented by replacing the spatial light modulator (2400) of the optoelectronic module (2010) shown in FIG. 16 with a smart pixel light modulator (2063). The smart pixel light modulator (2063) is formed to correspond to each of a plurality of first lenses (2062), and adjusts the intensity of light passing through the corresponding first lens (2062) through each pixel having a weight stored in a memory.

[0289] In addition, the smart pixel optical modulator (2063) is configured in a form in which a plurality of pixels are arranged, and a photodetector, an electronic processing unit, and a light source are assigned to each pixel. Here, the photodetector assigned to each pixel converts light passing through the first lens (2062) into a current signal, and the electronic processing unit amplifies the current signal or converts the result of calculating the current signal by multiplying or adding the current signal with a weight value stored in the memory into a current signal and outputs it to the connected light source.

[0290] In this way, the smart pixel optical modulator (2063) does not compromise the efficiency of parallel computation because each pixel is assigned a photodetector, an electronic processing unit, and a light source, so it does not need to be connected to the surrounding pixels. In addition, by using the smart pixel optical modulator (2063), the update speed of the existing weights can be improved from the kHz level to the hundreds of MHz level. This increase in the weight update speed is very effective when performing an algorithm that calculates optical convolution by applying various kernel sets to the same input data. While existing optical computers configure multiple optical channels as shown in Figure 14 to apply different kernel sets, the smart pixel can process this in a single channel using memory and a fast update speed. In addition, since various kernel sets can analyze various image features, they are widely used in image recognition algorithms. Therefore, when executing such algorithms, the parallel processing speed of the optical convolution neural network computer can be significantly improved and the volume of the optical module hardware can be reduced.

[0291] FIG. 21 is a conceptual diagram of an optoelectronic module using a smart pixel optical modulator including a fan-out according to a sixth embodiment of the present invention, and FIG. 22 is a conceptual diagram of the smart pixel optical modulator of FIG. 21.

[0292] The illustrated optoelectronic module (2070) includes a plurality of first photodetectors (2071), a plurality of first smart pixel optical modulators (2073), a plurality of second lenses (2074), a plurality of second photodetectors (2075), a plurality of electronic processing units (2076), and a plurality of second smart pixel optical modulators (2077). In this way, the optoelectronic module (2070) is a structure that further simplifies the structure of FIG. 20, in which the first lens and the third lens are eliminated and instead, an electrical fan-out structure is included in the smart pixel optical modulator. Here, the arrangement of the second lenses (2074) can be integrated on a single substrate, which greatly simplifies the structure and reduces manufacturing difficulty and cost.

[0293] A more detailed structure of this smart pixel optical modulator is shown in Fig. 22.

[0294] The electrical fan-out structure is a structure in which electrical input signals are branched into multiple branches through wires, and is simpler than a structure in which electrical input signals are optically connected through lenses. Therefore, by applying the electrical fan-out structure (2027) to a smart pixel optical modulator as shown in FIGS. 21 and 22, the overall hardware structure of the optoelectronic module can be further simplified.

[0295] The smart pixel light modulator is configured with a plurality of pixels arranged in a form, and each pixel is assigned an electronic processing unit (2028) and a light source (2029).

[0296] An electrical input signal is distributed to each of the pixels through a fan-out structure (2027). That is, the input signal is connected to an electronic processing unit (2028) through the fan-out structure (2027), and the electronic processing unit (2028) converts the result of calculating the input signal with a weight value stored in a memory into a current signal and outputs the result to a light source (2029). Here, the calculation may be, for example, a multiplication calculation, and therefore, the intensity of the output light source of each pixel has a characteristic that is proportional to the product of the input signal and the weight value.

[0297] Fig. 23 is a conceptual diagram of a bidirectional optoelectronic module using a spatial light modulator according to the first embodiment of the present invention.

[0298] The illustrated optoelectronic module (2080) includes a plurality of first light sources (2081), a plurality of second light detectors (2089), a plurality of first lenses (2082), a plurality of spatial light modulators (2083), a plurality of second lenses (2084), a plurality of third lenses (2085), a plurality of first light detectors (2086), a plurality of electronic processing units (2087), and a plurality of second light sources (2088).

[0299] The above optoelectronic module (2080) has a structure in which a first photodetector (2086) and a second light source (2088) are arranged on the same substrate at the output terminal to enable bidirectional calculation. Light starting from the second light source sequentially passes through a third lens (2085), a second lens (2084), a spatial light modulator (2083), and the first lens (2082) and is collected at the second photodetector (2089) located next to the first light source (2081). Here, if the spatial light modulator (2083) is replaced with the smart pixel light modulator proposed in the present invention, a bidirectional optical neural network computer with a fast weight update speed can be implemented.

[0300] Fig. 24 is a conceptual diagram of a smart pixel optical modulator applied to the bidirectional optoelectronic module of Fig. 23.

[0301] The illustrated smart pixel optical modulator has a first light source (2032) and a first photodetector (2031) within the smart pixel arranged in each of the pixels arranged on the first side on the left. In addition, a second light source (2035) and a second photodetector (2034) within the smart pixel are arranged in each of the pixels arranged on the second side on the right. In addition, an electronic processing unit (2033A, 2033B) is arranged in each of the pixels arranged in the middle between the first side and the second side.

[0302] A pixel that sends light incident on the first side to the second side is electrically connected to a first photodetector (2031) within a smart pixel, an electronic processing unit (2033B) of the pixel, and a second light source (2035) within the smart pixel. A pixel that sends light incident on the second side to the first side is electrically connected to a second photodetector (2034) within a smart pixel, an electronic processing unit (2033A) of the pixel, and a first light source (2032) within the smart pixel. At this time, in order to allow the light source of the first side to exit at a specific angle, a micro prism may be installed on the side where the light source exits, and a micro lens may be installed in front of the first light source or the second light source within the smart pixel to adjust the divergence angle of the output beam.

[0303] FIG. 25 is a conceptual diagram of a bidirectional optoelectronic module using a smart pixel optical modulator including fan-in and fan-out according to a second embodiment of the present invention, and FIG. 26 is a conceptual diagram of the smart pixel optical modulator of FIG. 25.

[0304] The illustrated optoelectronic module (2090) includes a plurality of smart pixel optical modulators (2091, 2097), a plurality of second lenses (2092), a plurality of photodetectors (2094), a plurality of light sources (2095), and a plurality of electronic processing units (2096). In this way, the optoelectronic module (2090) has a simplified structure by applying a smart pixel optical modulator to the structure of FIG. 23. That is, the first lens (2082) and the third lens (2085) illustrated in FIG. 23 are eliminated, and a smart pixel optical modulator, such as that in FIG. 26, in which a fan-out and fan-in structure, a light source, a photodetector, and an electronic processing unit are integrated is used. This structure simplifies the hardware structure and greatly increases the parallel processing speed.

[0305] A more detailed structure of the smart pixel optical modulator is illustrated in FIG. 26. The smart pixel includes both a light source (2045) and a photodetector (2044) on the second surface, and includes both a fan-out and fan-in structure (2041, 2042) connecting the electronic processing unit (2043) and the electrical input / output nodes.

[0306] The electrical fan-in structure (2042) calculates the signal acquired from the photodetector (2044) with the weights of the memory, converts the result value into a current, and transmits it to the input / output node, and the electrical fan-out structure (2041) distributes the input signal to multiple branches through wires and transmits it. This structure enables the smart pixel optical modulator to operate in both directions and enables the optical neural network computer to compute algorithms such as the backpropagation algorithm. Since the smart pixel optical modulator has a fast switching speed and does not harm parallelism, it greatly improves the parallel processing capacity of the bidirectional optical neural network computer.

[0307] Figure 27 is a conceptual diagram explaining the backpropagation algorithm in an artificial neural network. Weighted input This weight class It is used between, and this feature theoretically has the advantage that the development of the theory is easier when using weighted input. The first layer ( ) node value is the weight Multiply by and then bias Add weighted input and construct a sigmoid function Enter the output value of the next layer node This is shown in the following mathematical expressions 5 and 6.

[0308] [Equation 5]

[0309]

[0310] [Equation 6]

[0311]

[0312] Assuming that the second layer is the last layer of the neural network, the target value of each node during learning is Then, the sum of the squares of the differences between the output value of the node and the target value can be defined as the error or merit function E as in mathematical expression 7.

[0313] [Equation 7]

[0314]

[0315] The principle of learning in artificial neural networks is the process of adjusting weights and biases or weighted inputs to minimize the error value. In this process, a correction value is added to the weights or weighted inputs. The steepest gradient method, which is one of the minimization algorithms, is used, which views the weights and weighted inputs as variables, calculates the gradient vector for these variables, changes the sign, and then adds the correction value in that direction. In artificial intelligence theory, this gradient vector is also called a sensitivity vector, and here, is expressed as . In addition, this sensitivity is proportional to the weights, bias, error, or correction amount of the weighted input. The sensitivity vector for the weighted input of the first layer is shown in the following mathematical expression 8.

[0316] [Equation 8]

[0317]

[0318] By applying the chain rule of partial differentiation and equations 5 and 7, If we obtain , we can obtain the following mathematical equation 9. The final output error is multiplied by a function that differentiates the sigmoid function.

[0319] [Equation 9]

[0320]

[0321] The relationship between the sensitivity of the first layer and the sensitivity of the second layer can also be obtained by applying the chain rule and equations 5 and 6. The result is shown in equation 10 below. Adjacent weighted inputs are connected by weights and a sigmoid differential function. Ultimately, this structure, in the form of a recurrence relation, allows us to derive the weighted input of the first layer if we know the weighted input of the second layer. Equation 9 provides the initial value of the recurrence relation.

[0322] [Equation 10]

[0323]

[0324] Similarly, the sensitivity of the weights can be calculated using the chain rule and Equation 6, and the result is shown in Equation 11 below. The sensitivity of the weights is equal to the product of the sensitivity of the weighted inputs of the subsequent layer and the node value of the previous layer. Using the weighted input sensitivity vector of each layer, obtained using the recurrence relation, the sensitivity of the weights can also be calculated.

[0325] [Equation 11]

[0326]

[0327] The bias sensitivity can also be obtained in the same way, and is equal to the value of the weight sensitivity assuming that the input node value is 1. The result is shown in Equation 12 below.

[0328] [Equation 12]

[0329]

[0330] In this way, the weight and bias sensitivity vectors of each layer can be obtained. By continuously modifying the sensitivity vectors by changing the sign of the sensitivity, multiplying them by a certain coefficient, and adding them to the previous value, the error function can be minimized. Applying this backpropagation algorithm to the general (l-1)th layer, the lth layer, and the final Lth layer yields Equations 13 to 17 below.

[0331] [Equation 13]

[0332]

[0333] [Equation 14]

[0334]

[0335] [Equation 15]

[0336]

[0337] [Equation 16]

[0338]

[0339] [Equation 17]

[0340]

[0341] As shown in Equation 15, the weighted input sensitivity of the preceding layer is the weighted value added to the sensitivity of the subsequent layer, which is basically similar to the forward artificial neural network calculation except that the direction of propagation is changed. Therefore, the lens array or spatial light modulator array used in existing optical neural network computers can be used similarly. However, sending light in both the forward and reverse directions requires additional light source arrangements, detector arrangements, and securing an optical path for reverse propagation, which necessitates a new system structure, such as a structural change in the light source.

[0342] Fig. 28 is a conceptual diagram of a non-differential mode bidirectional optoelectronic module using a spatial light modulator according to a third embodiment of the present invention.

[0343] An optoelectronic module (3050, hereinafter referred to as an optoelectronic module) using a spatial light modulator according to the present invention operates in parallel by combining a light source, a lens, a spatial light modulator, a light detector, etc. instead of a complex electronic circuit. The optoelectronic module (3050) according to the present invention can be applied to an optical neural network computer.

[0344] Referring to FIG. 28, an optoelectronic module (3050) according to one embodiment of the present invention includes a plurality of first light sources (3401a, 3401b), a plurality of first lenses (3402a, 3402b), a plurality of spatial light modulators (3403a, 3403b), a plurality of first light detectors (3420a, 3420b), a plurality of third light sources (3424a, 3424b), a plurality of third light detectors (3440a, 3440b), and a lens unit (3404a, 3404b, 3405).

[0345] A plurality of first light sources (3401a, 3401b) may be formed at regular intervals on the first substrate (3400), but may also be formed on different substrates.

[0346] The optical optoelectronic module (3050) according to the third embodiment of the present invention may further include a plurality of electronic processing units (3421a, 3421b, 3441a, 3441b), a plurality of second light sources (3422a, 3422b), and a plurality of second light detectors (3460a, 3460b).

[0347] A plurality of first and second photodetectors (3420a, 3420b, 3460a, 3460b), a plurality of electronic processing units (3421a, 3421b), a plurality of second light sources (3422a, 3422b), and a plurality of third light sources (3424a, 3424b) may be formed on a second substrate (3430), but may alternatively be formed separately on another substrate or one or more elements may be formed on the same substrate. For example, the first substrate (3400) and the second substrate (3430) may be semiconductor substrates or PCBs (Printed Circuit Boards).

[0348] A plurality of first lenses (3402a, 3402b) can be formed on the same substrate.

[0349] Multiple spatial light modulators (3403a, 3403b) can be formed on the same substrate.

[0350] A plurality of second lenses (3404a, 3404b) can be formed on the same substrate.

[0351] A plurality of first light sources (3401a, 3401b) and third light detectors (3440a, 3440b) can be formed on the same substrate.

[0352] A plurality of third light sources (3424a, 3424b) and first light detectors (3420a, 3420b) can be formed on the same substrate.

[0353] In FIG. 28, the first light source (3401a, 3401b), the first lens (3402a, 3402b), the spatial light modulator (3403a, 3403b), the first photodetector (3420a, 3420b), the third light source (3424a, 3424b), the third photodetector (3440a, 3440b), the second light source (3422a, 3422b), the second photodetector (3460a, 3460b), etc. are expressed as two, and the electronic processing unit (3421a, 3421b, 3441a, 3441b) is expressed as four, but these are only examples, and the number of each element can be changed and designed as needed.

[0354] For example, in the present invention, the light source may be a light emitting diode (LED) or a semiconductor laser, particularly a vertical-cavity surface-emitting laser. The intensity of light can be controlled by varying the intensity of the current flowing through the light source, thereby individually controlling the input values.

[0355] Light emitted from a plurality of first light sources (3401a, 3401b) passes through a plurality of corresponding first lenses (3402a, 3402b), and then passes through a corresponding spatial light modulator (3403a, 3403b). The first lenses (3402a, 3402b) reduce the divergence angle of the light emitted from the corresponding first light sources (3401a, 3401b), and preferably modulate the light into parallel light.

[0356] In Fig. 28, four representative rays (3410a, 3412a, 3410b, 3412b) from among the light emitted from each first light source (3401a, 3401b) are shown, and the four rays pass through individual pixels in a spatial light modulator (3403a, 3403b). The spatial light modulators (3403a, 3403b) are formed corresponding to each of the plurality of first lenses (3402a, 3402b), and adjust the intensity of the rays passing through the corresponding first lenses (3402a, 3402b) through respective pixels having preset weights.

[0357] For example, liquid crystal display panels (LCDs) can be used as each spatial light modulator (3403a, 3403b). Since LCD panels regulate the intensity of incoming light to different transmittances according to voltage and allow them to pass through, the intensity of each light beam can be adjusted to a predetermined value.

[0358] Therefore, even if the light originates from the same light source, the intensity of the light passing through each pixel on the LCD panel varies depending on the transmittance of each pixel. For example, if the illuminance of the light source is 1.0 W / m 2 In this case, if the transmittance of the LCD pixel is 0.10, the illuminance of the output light is 0.10 W / m 2 And if the transmittance is 0.05, the illuminance of the output light is 0.05 W / m 2 It becomes.

[0359] The transmittance of the LCD panel can be adjusted in more than 256 steps depending on the voltage, so in this way, the product of the weights and input values ​​in the neural network can be implemented. Here, the input value is the intensity of the light source and the transmittance of the spatial light modulator corresponds to the weight.

[0360] Light rays passing through the spatial light modulators (3403a, 3403b) are input to the lens units (3404a, 3404b, 3405). The lens units (3404a, 3404b, 3405) are formed between a plurality of spatial light modulators (3403a, 3403b) and a plurality of first photodetectors (3420a, 3420b).

[0361] Additionally, the lens units (3404a, 3404b, 3405) serve to collect light rays irradiated from different first light sources (3401a, 3401b) passing through pixels at the same relative positions of each spatial light modulator (3403a, 3403b) onto one first light detector (3420a, 3420b).

[0362] In an embodiment of the present invention, the lens units (3404a, 3404b, 3405) are formed corresponding to the plurality of spatial light modulators (3403a, 3403b), respectively, and include a plurality of second lenses (3404a, 3404b) that focus light rays to a single point, and a third lens (3405) that passes all light rays that have passed through the plurality of second lenses (3404a, 3404b). If the light rays incident on the second lenses (3404a, 3404b) are parallel light rays, they are focused at a focus.

[0363] The light rays passing through each spatial light modulator (3403a, 3403b) pass through the corresponding second lens (3404a, 3404b), are focused, and then spread out again, where they pass through the third lens (3405) again.

[0364] When a ray passes through the second lens (3404a, 3404b), the angle of the ray changes depending on the height of the ray away from the optical axis of the second lens (3404a, 3404b). The ray (3410a) passing through the highest position on the optical axis of the second lens (3404a) and the ray (3410b) passing through the highest position on the optical axis of the second lens (3404b) have the same angle with respect to the horizontal axis.

[0365] The two light rays (3410a, 3410b) enter the third lens (3405) while forming parallel light, and thus converge at the same point on the focal plane of the third lens (3405). In Fig. 28, the two light rays converge on one first photodetector (3420a).

[0366] Similarly, light rays (3412a) passing through the second lens (3404a) and light rays (3412b) passing through the second lens (3404b), which are rays at relatively the same position, pass through the third lens (3405) and are then collected on one photodetector (3420b).

[0367] The first photodetector (3420a, 3420b) can be composed of an optical semiconductor element, such as a photodiode. The current flowing in the first photodetector (3420a, 3420b) is ultimately the sum of the intensities of light arriving at each of the first photodetectors (3420a, 3420b). , It has a value proportional to the current value obtained from the first photodetector (3420a, 3420b) or its amplified value and the bias value ( stored in the electronic processing unit (3421a, 3421b) respectively. , ) is added, and the sigmoid function б is calculated using that value as input, and the output value of the neural network node is obtained.

[0368] The rays (3411a, 3413a, 3411b, 3413b) show examples of rays traveling in the reverse direction. If the optical module operates in the forward direction, the rays from the first light source (3401a, 3401b) can also enter and travel in the forward direction, and in this case, the pixels , , , By setting it to the OFF state, the light rays from the first light source (3401a, 3401b) can not pass through. In fact, the reverse light rays (3411a, 3413a, 3411b, 3413b) start from the first photodetector (3420a, 3420b) and reach the first light source (3401a, 3401b), so they do not generate a signal and are an example of the reverse light path.

[0369] In one embodiment, each output value of the electronic processing unit (3421a, 3421b) may be transmitted to the connected second light source (3422a, 3422b), respectively. Accordingly, each output value of the electronic processing unit (3421a, 3421b) , can be adjusted proportionally.

[0370] Each electronic processing unit (3421a, 3421b) can use other activation functions, such as the RELU function, in addition to the sigmoid function.

[0371] In addition to performing function operations, the electronic processing units (3421a, 3421b) can store result values ​​or external input values ​​for use in calculations. This memory function enables various calculations by comparing calculated values ​​in the next sequence with stored values ​​or by using stored values ​​for function calculations.

[0372] The electronic processing unit has a communication function and can exchange data with surrounding electronic processing units or external processors through connected communication lines.

[0373] The second light source (3422a, 3422b) is the neural network. , It corresponds to and is used as the input value for the next layer.

[0374] The electronic processing unit (3421a, 3421b) is an electronic integrated circuit formed on a semiconductor substrate and can be implemented through an analog circuit or a digital circuit.

[0375] The lenses used in this embodiment can be implemented using refractive lenses or diffractive optical elements.

[0376] The first light source (3401a, 3401b) and the first lens (3402a, 3402b) corresponding thereto can be formed to be spaced apart by the focal length of the first lens (3402a, 3402b) within an error of 20%.

[0377] In addition, the spatial light modulators (3403a, 3403b) and the second lenses (3404a, 3404b) corresponding thereto, respectively, may be formed to be spaced apart from each other by the focal length of the second lenses (3404a, 3404b) within an error of 20%, and the first photodetectors (3420a, 3420b) may be formed to be spaced apart from the third lens (3405) by the focal length of the third lens (3405) within an error of 40%.

[0378] In particular, when the spatial light modulator (3403a, 3403b) is spaced apart from the second lens (3404a, 3404b) by the focal length of the second lens (3404a, 3404b), and the lens (3405) is spaced apart from the photodetector (3420a, 3420b) by the focal length of the third lens (3405), the image of the pixel of the spatial light modulator (3403a, 3403b) is generated on the first photodetector, so that cross-talk can be reduced.

[0379] However, even if the distance between the spatial light modulator (3403a, 3403b) and the second lens (3404a, 3404b) is not exactly the focal length but slightly off, the distance between the third lens (3405) and the first photodetector (3420a, 3420b) can be adjusted so that the image of the pixel of the spatial light modulator (3403a, 3403b) is formed on the first photodetector (3420a, 3420b), so that a certain degree of tolerance can be achieved.

[0380] Likewise, the distance between the first light source (3401a, 3401b) and the corresponding first lens (3402a, 3402b) can be adjusted to maintain performance even if the distance between other lenses or elements deviates slightly from the focal length.

[0381] For example, a plurality of first lenses (3402a, 3402b) corresponding to a plurality of light sources can be formed on a single substrate, a plurality of spatial light modulators (3403a, 3403b) can also be formed on a single substrate, and a plurality of second lenses (3404a, 3404b) can also be formed on a single substrate. This structure can facilitate the volume and optical alignment of the entire system.

[0382] In the above description, the forward operation of the artificial neural network optical module, in which light originates from the first light source (3401a, 3401b) of the first substrate (3400) and arrives at the first photodetector (3420a, 3420b) on the second substrate (3430), has been described. For reverse light propagation and data processing, an optical module structure is required in which light originates from the third light source (3424a, 3424b) on the second substrate (3430) and arrives at the third photodetector (3440a, 3440b) on the first substrate (3400).

[0383] Figure 28 shows rays (3415a, 3415b, 3416a, 3416b) originating from a third light source (3424a, 3424b). Rays (3411a, 3413a, 3411b, 3413b), indicated by thin lines, take a similar path to forward rays (3410a, 3412a, 3410b, 3412b) and correspond to reverse rays, but are used here as auxiliary lines to explain the reverse main rays (3415a, 3415b, 3416a, 3416b). If the light beam originating from the actual third light source (3424a, 3424b) proceeds to the auxiliary line (3411a, 3413a, 3411b, 3413b), it will reach the first light source (3401a, 3401b) and thus cannot generate a signal from the third light detector (3440a, 3440b). However, if the main rays (3415a, 3415b, 3416a, 3416b) start in an upward direction compared to the auxiliary rays (3411a, 3413a, 3411b, 3413b), the two rays meet at an angle that is not parallel to the lens optical axis in the spatial light modulator (3403a, 3403b), and the main rays (3415a, 3415b, 3416a, 3416b) arrive at the third photodetector (3440a, 3440b) located below the first light source (3401a, 3401b). The reason why the two light rays (3411a, 3415a) meet at the spatial light modulator is because the spatial light modulator (3403a, 3403b) and the third light source (3424a, 3424b) are in a distance relationship with the object.

[0384] When the main light beam (3415a, 3415b, 3416a, 3416b) starts downward compared to the auxiliary light beam (3411a, 3413a, 3411b, 3413b), it passes through the upper part of the first light source (3401a, 3401b) on the surface where the first substrate (3400) is located, and the third light detector (3440a, 3440b) must be located above the first light source (3401a, 3401b) to detect the light. In the case of the electronic processing unit (3421a), the node value , , Because it is calculating or storing can be obtained. The bias sensitivity or correction value is given by Equation 13. can be obtained through. The third light source (3424a) is connected to the electronic processing unit (3421a). The intensity of light proportional to the pixel of the spatial light modulator (3403a) can be sent. and the intensity of the light passes through is proportional to . Here and The two pixels corresponding to are spatially separated but set to the same value.

[0385] Likewise, the primary ray (3416a) originating from the third light source (3424b) has light proportional to the pixels of the spatial light modulator (3403a) and the intensity of light passes through is proportional to. These two rays (3415a, 3416a) meet and add at the third photodetector (3440a). is converted into an electrical signal proportional to the electronic processing unit (3441a). Multiply by can be obtained in the same way. can be obtained.

[0386] Additionally, near each pixel of the spatial light modulator (3403a), a photodetector and an electronic processing unit are preferably formed within the same substrate and connected to each pixel to measure the intensity of light passing through them, store data, exchange information with surrounding pixels, and perform mathematical operations. and In an electronic processing unit close to and Because we can obtain weight sensitivity can be obtained in the same way. , , can also be obtained. Ultimately, the optoelectronic module (3050) can perform artificial neural network calculations not only in the forward direction but also in the reverse direction, and is capable of large-scale parallel calculations.

[0387] Figure 29 is a conceptual diagram of the third light source of Figure 28 implemented using a diffraction grating and a prism.

[0388] Referring to FIG. 29, a light beam starting from a third light source (3424a) is separated into several diffracted light beams as it passes through a diffraction grating (3450a), and its direction is bent as it passes through a prism (3451a). The angle at which the light beams are bent is determined according to the inclination angle of the prism (3451a), and the angle between the diffracted light beams is determined according to the period of the diffraction grating (3450a). The third light source (3424a) can be a semiconductor light source such as an LED (Light Emitting Diode) or a VCSEL (Vertical Cavity Surface Emitting Laser), and a lens can be added between the third light source (3424a) and the diffraction grating (3450a) to adjust the divergence angle. The diffraction grating (3450a), the prism (3451a), and the lens can be replaced with a diffractive optical element.

[0389] Fig. 30 is a drawing showing an example of cascading the optoelectronic modules of Fig. 28.

[0390] Referring to FIG. 30, the first optical computation layer (3801) is repeated and appears in the second optical computation layer (3802), and an arbitrary number of neural network layers can be implemented. The output value from the first optical computation layer (3801), which is the preceding layer, can be connected to the input of the second optical computation layer (3802), which is the succeeding layer.

[0391] Another method of constructing a multilayer neural network in FIG. 28 is to perform multilayer neural network calculations by having light travel back and forth between the first substrate (3400) and the second substrate (3430) and storing the calculation results in electronic processing units (3421a, 3421b, 3441a, 3441b). At this time, each time the light propagates in one direction, the weight values ​​of the spatial light modulators (3403a, 3403b) must be changed according to the values ​​of each layer. At this time, the weight pixels are updated by retrieving the values ​​stored in the memory of the electronic processing units connected to the pixels.

[0392] Fig. 31 is a conceptual diagram of a differential mode bidirectional optoelectronic module using a spatial light modulator according to the fourth embodiment of the present invention.

[0393] The optoelectronic module (3070) using the spatial light modulator according to the present embodiment can be implemented with substantially the same configuration as the optoelectronic module (3050) of FIG. 28, except for the configuration of the light source and the light detector for implementing differential mode calculation. Therefore, a repeated description of the same components as the optoelectronic module (3050) of FIG. 28 is omitted.

[0394] Referring to FIG. 31, an optical optoelectronic module (3070) according to a fourth embodiment of the present invention includes a plurality of first A photodetectors (3610a, 3610c), a plurality of first B photodetectors (3610b, 3610d), a plurality of third light sources (3620a, 3620b, 3620c, 3620d), a plurality of third A photodetectors (3540a, 3540c), and a plurality of third B photodetectors (3540b, 3540d). In addition, the module may further include a plurality of electronic processing units (3630a, 3630b, 3550a, 3550b) and corresponding second light sources (3701a, 3701b).

[0395] The output values ​​of the first A photodetector (3610a) and the first B photodetector (3610b) may be connected to an electronic processing unit (3630a), and the output values ​​of the first A photodetector (3610c) and the first B photodetector (3610d) may be connected to an electronic processing unit (3630b).

[0396] The structure of this embodiment provides the advantage that the electronic processing units (3630a, 3630b) can perform various operations using the values ​​of the two photodetectors. The easiest example is subtraction. For example, Let us assume a case of calculation. Here, superscripts are omitted to avoid the complexity of representing layers in the weight expression. Therefore, Is It represents.

[0397] Subtraction using light is possible using principles like destructive interference, but this increases the complexity of optical computing circuits, as it requires adjusting for path differences. Subtraction is not easy when two light beams enter a photodetector that only detects light intensity.

[0398] Therefore, in this embodiment, light corresponding to the (+) term is collected by the 1A photodetector (3610a, 3610c), light corresponding to the (-) term is collected by the 1B photodetector (3610b, 3610d), and the electronic processing unit (3630a, 3630b) calculates by subtracting the two signal intensities.

[0399] In the weight expression in Fig. 31 and Is and Represents the positive part of the weight and Is and Represents the negative part of the weight. For example, is positive If is negative, , , Using You can get results.

[0400] Even if there are any number of terms, the rays corresponding to the (+) terms are weighted Set the original model value, and the rays corresponding to the (-) term have weights After setting to 0, only the rays corresponding to the (+) term can be collected and added to the photodetector (610a). Similarly, the rays corresponding to the (-) term can be collected and sent to the photodetector (3610b) for addition. Therefore, in this way, general neural network operations including negative weights can be performed.

[0401] Each electronic processing unit (3630a, 3630b) can use the output of not only two surrounding photodetectors but also a plurality of adjacent photodetectors as input, and can implement various functions other than subtraction through electronic circuits.

[0402] In practice, subtraction and sigmoid function operations can be implemented using analog or digital electronic circuits. While this embodiment illustrates connections between multiple photodetectors and electronic processing units, connections between multiple electronic processing units can also achieve the same effect.

[0403] Each electronic processing unit (3630a, 3630b) can also store a bias value, amplify the current input from the photodetector, or convert the result of the operation into a current value proportional to the result when sending it to the input of the second light source.

[0404] In order to enable such differential mode neural network calculations in the reverse direction, an additional third light source (3620a, 3620b, 3620c, 3620d), multiple third A photodetectors (3540a, 3540c), and multiple third B photodetectors (3540b, 3540d) are required, as shown in FIG. 31. Light rays (3621a, 3621b, 3622a, 3622b, 3623a, 3623b, 3624a, 3624b) starting from the third light source (3620a, 3620b, 3620c, 3620d) sequentially pass through the third lens (3534), the second lens (3532a, 3532b), the spatial light modulator (3531a, 3532b), and the first lens (3530a, 3530b) and are then gathered on the thirdA light detector (3540a, 3540c) and the thirdB light detector (3540b, 3540d).

[0405] Light rays (3621a, 3621b, 3622a, 3622b) originating from the third light source (3620a, 3620b) are connected to the electronic processing unit (3630a) and input. has an intensity proportional to the third light source (3620a). The intensity of light coming from the third light source (3620b) adjacent to the third light source (3620a) is shown in Fig. 31. and are marked as and have the same value has a size proportional to . Similarly, the light from the other third light source (3620c, 3620d) and are marked as and have the same value has a century proportional to it.

[0406] The third light source (3620a, 3620b) is a pixel of the spatial light modulator (3531a). , ) and the lens unit (3532a, 3534) correspond one-to-one as an object-image relationship. If the spatial light modulator (3531a) is spaced apart from the second lens (3532a) by the focal length of the second lens and the third lens (3534) is spaced apart from the third light source (3620a, 3620b) by the focal length of the third lens, this object-image relationship is established, and if the distance between the spatial light modulator (3531a) and the second lens (3532a) changes, the object-image relationship can be satisfied by adjusting the distance between the third lens (3534) and the third light source (3620a, 3620b). Therefore, the light rays (3621a, 3622a) departing from the third light source (3620a, 3620b) are focused on the pixels ( , ) and the spatial light modulator (3531a) and face downward and upward, respectively. This direction is related to the angle of the light rays departing from the third light source (3620a, 3620b).

[0407] If the angle of the departing light beam is slightly upward, the downward angle increases when passing through the spatial light modulator (3531a). Conversely, if the angle of the departing light beam is downward compared to the original angle, the angle of the light beam changes to an upward angle compared to the original angle when passing through the spatial light modulator (3531a). This angle adjustment can be adjusted through the grating period of the prism (3451a) or the diffraction grating (3450a) used in the third light source (3620a, 3620b). The angle of the light beam (3621a, 3622a) when passing through the spatial light modulator (3531a) determines the height of the light beam when it reaches the substrate (3500) and allows the two light beams (3621a, 3622a) to reach the third A photodetector (3540a) and the third B photodetector (3540b), respectively.

[0408] In the same way, the light rays (3623a, 3624a) starting from the third light source (3620c, 3620d) reach the third A photodetector (3540a) and the third B photodetector (3540b). Therefore, the signal detected by the third photodetector (3540a) is The signal detected by the 3B photodetector (3540b) is proportional to is proportional to. The electronic processing unit (3550a) connected to these two photodetectors can perform a subtraction operation with the magnitude of these two signals. A proportional signal size can be obtained.

[0409] As with the forward differential mode calculation above, in the reverse weight representation in Fig. 31, and Is and Represents the positive part of the reverse weight, and Is and Represents the negative part of the reverse weight. For example, is positive If is negative, = , =0, =0, = Using You can get the fruit.

[0410] [Equation 18]

[0411]

[0412] Even in the reverse calculation, if there are any number of terms as in Equation 18, the rays corresponding to the (+) term are weighted Set the original model value, and the rays corresponding to the (-) term have weights After setting to 0, only the rays corresponding to the (+) term can be collected and added to the 3A photodetector (3540a). Similarly, the rays corresponding to the (-) term can be collected and sent to the 3B photodetector (3540b) for addition. Therefore, in this way, general neural network operations including negative weights can be performed.

[0413] Each electronic processing unit (3550a, 3550b) can use the output of not only two surrounding photodetectors but also a number of adjacent photodetectors as input, and can implement various functions other than subtraction through electronic circuits.

[0414] Input value in equation 18 in reverse direction Instead, the weighted input stored by the electronic processing unit (3550a) can be used can be obtained. The electronic processing unit (3550a) uses the values ​​being calculated or stored. Here silver The pixels corresponding to are spatially separated but set to the same value. In the same way can be obtained.

[0415] Additionally, the spatial light modulator (3531a) has a photodetector and an electronic processing unit connected to each pixel to measure the intensity of light passing through it, store data, exchange information with surrounding pixels, and perform mathematical operations. and In an electronic processing unit close to and Because we can obtain weight sensitivity can be obtained in the same way. , , , , In this way, the optoelectronic module (3070) can ultimately perform artificial neural network calculations with an arbitrary number of input / output nodes or terms in the forward as well as reverse direction, and can process large amounts of parallel data.

[0416] Fig. 32 is a drawing showing an example of cascading the optoelectronic modules of Fig. 31.

[0417] Referring to FIG. 32, the first optical computation layer (3901) is repeated and appears in the second optical computation layer (3902), and an arbitrary number of neural network layers can be implemented. The output value of the first optical computation layer (3901), which is the preceding layer, can be connected to the input of the second optical computation layer (3902), which is the succeeding layer.

[0418] Another method of constructing a multilayer neural network in FIG. 31 is to perform multilayer neural network calculations by having light travel back and forth between the first substrate (3500) and the second substrate (3700) and storing the calculation results in electronic processing units (3630a, 3630b, 3550a, 3550b). In this case, each time the light propagates in one direction, the weight values ​​of the spatial light modulators (3531a, 3532b) must be changed according to the values ​​of each layer. The electronic processing units connected to the weight pixels retrieve and update the values ​​stored in the memory.

[0419] The present invention provides an optical neural network computer structure and technology that increases the computational speed by performing approximately NXM multiplications and additions at once without complicating the physical structure of the connection lines or generating noise as in an electrical circuit even when the number of input and output terminals increases to N and M, respectively, by implementing an optical computer structure using optical connections instead of purely electronic circuits.

[0420] When the pixel values ​​of the spatial light modulator are already determined, parallel computation is possible because the optical paths from the light source to the photodetector overlap, but photons on different paths do not interfere with each other, and the computation time is very short because photons arrive at the photodetector at the speed of light.

[0421] The optical neural network computer proposed in the present invention can perform and apply various algorithms, such as neural network learning algorithms such as backpropagation algorithms or auto-correlation algorithms, as well as inference calculations, because light can propagate in both directions.

[0422] In addition, the time delay in the calculation occurs in the electronic processing unit, but since it is performed in parallel in all outputs, it can be considered as one instruction cycle of the digital electronic circuit. When configuring L multilayer neural networks, the calculation is performed simultaneously in all layers, so the actual calculation speed increases by NXMXL times. In the previously described Figures 28 and 29, a bidirectional optoelectronic module using a spatial light modulator and an optical neural network computer including the same were described.

[0423] However, these optical neural network computers utilize slow-switching spatial light modulators, which can be time-consuming to update weight sets when performing calculations on different sets of weights for the same input data. Furthermore, the need to update these weight sets and the slow switching speed of the spatial light modulator can hinder the parallel processing speed of optical neural network computers.

[0424] Therefore, below, a bidirectional optoelectronic module using a smart pixel optical modulator and an optical neural network computer including the same will be described through FIGS. 33 to 37.

[0425] FIG. 33 is a conceptual diagram of a bidirectional optoelectronic module using a smart pixel optical modulator according to a fifth embodiment of the present invention, and FIG. 34 is a conceptual diagram of the smart pixel optical modulator of FIG. 33.

[0426] The illustrated optoelectronic module (3010) includes a plurality of first light sources (3101a, 3101b), a plurality of first lenses (3102a, 3102b), a plurality of smart pixel light modulators (3103a, 3103b), a plurality of first light detectors (3120a, 3120b), a plurality of third light sources (3124a, 3124b), a plurality of third light detectors (3140a, 3140b), and a lens unit (3104a, 3104b, 3105).

[0427] A plurality of first light sources (3101a, 3101b) may be formed at regular intervals on the first substrate (3100), but may also be formed on different substrates.

[0428] The optical optoelectronic module (3010) according to the fifth embodiment of the present invention may further include a plurality of electronic processing units (3121a, 3121b, 3141a, 3141b), a plurality of second light sources (3122a, 3122b), and a plurality of second light detectors (3160a, 3160b).

[0429] A plurality of first and second photodetectors (3120a, 3120b, 3160a, 3160b), a plurality of electronic processing units (3121a, 3121b), a plurality of second light sources (3122a, 3122b), and a plurality of third light sources (3124a, 3124b) may be formed on a second substrate (3130), but may alternatively be formed separately on another substrate or one or more elements may be formed on the same substrate. For example, the first substrate (3100) and the second substrate (3130) may be semiconductor substrates or PCBs.

[0430] A plurality of first lenses (3102a, 3102b) can be formed on the same substrate.

[0431] Multiple smart pixel optical modulators (3103a, 3103b) can be formed on the same substrate.

[0432] A plurality of second lenses (3104a, 3104b) can be formed on the same substrate.

[0433] A plurality of first light sources (3101a, 3101b) and third light detectors (3140a, 3140b) can be formed on the same substrate.

[0434] A plurality of third light sources (3124a, 3124b) and first light detectors (3120a, 3120b) can be formed on the same substrate.

[0435] In FIG. 33, the first light source (3101a, 3101b), the first lens (3102a, 3102b), the smart pixel light modulator (3103a, 3103b), the first light detector (3120a, 3120b), the third light source (3124a, 3124b), the third light detector (3140a, 3140b), the second light source (3122a, 3122b), the second light detector (3160a, 3160b), etc. are expressed as two, and the electronic processing unit (3121a, 3121b, 3141a, 3141b) is expressed as four, but these are only examples, and the number of each element can be changed and designed as needed.

[0436] For example, in the present invention, the light source may be an LED or a semiconductor laser, particularly a vertical surface-emitting laser. The intensity of light can be adjusted by varying the intensity of the current flowing through the light source, thereby individually controlling the input values.

[0437] Light emitted from a plurality of first light sources (3101a, 3101b) passes through a plurality of corresponding first lenses (3102a, 3102b), and then passes through a corresponding smart pixel light modulator (3103a, 3103b). The first lenses (3102a, 3102b) reduce the divergence angle of the light emitted from the corresponding first light sources (3101a, 3101b), and preferably modulate the light into parallel light.

[0438] In Fig. 33, four representative light rays (3110a, 3112a, 3110b, 3112b) are shown among the light emitted from each first light source (3101a, 3101b), and the four light rays pass through individual pixels in a smart pixel light modulator (3103a, 3103b). The smart pixel light modulator (3103a, 3103b) is formed corresponding to each of the plurality of first lenses (3102a, 3102b), and adjusts the intensity of the light rays passing through the corresponding first lenses (3102a, 3102b) through each pixel having a weight set by reading from a pixel memory.

[0439] In addition, the smart pixel light modulator (3103a, 3103b) is configured in a form in which a plurality of pixels are arranged, and each pixel is assigned a light detector, an electronic processing unit, and a light source. Here, the light detector converts light passing through the first lens (3102a, 3102b) or the second lens (3104a, 3104b) into a current signal, and the electronic processing unit amplifies the current signal or converts the result of an operation such as multiplication or addition of the current signal with a weight value stored in a memory into a current signal and outputs the result to the connected light source.

[0440] The structure of the smart pixel light modulator (3103a, 3103b) will be described in more detail with reference to FIG. 34. The illustrated smart pixel light modulator has a first light source (3172) and a first photodetector (3171) within the smart pixel arranged in each of the pixels arranged on the first side on the left. In addition, a second light source (3175) and a second photodetector (3174) within the smart pixel are arranged in each of the pixels arranged on the second side on the right. In addition, an electronic processing unit (3173A, 3173B) is arranged in each of the pixels arranged in the middle between the first side and the second side.

[0441] A pixel that sends light incident on the first side to the second side is electrically connected to a first photodetector (3171) within a smart pixel, an electronic processing unit (3173B) of the pixel, and a second light source (3175) within the smart pixel. A pixel that sends light incident on the second side to the first side is electrically connected to a second photodetector (3174) within a smart pixel, an electronic processing unit (3173A) of the pixel, and a first light source (3172) within the smart pixel. At this time, in order to allow the light source of the first side to exit at a specific angle, a micro prism may be installed on the side where the light source exits, and a micro lens may be installed in front of the first light source or the second light source within the smart pixel to adjust the divergence angle of the output beam.

[0442] In this way, the smart pixel light modulator does not compromise the efficiency of parallel computation because each pixel is assigned a photodetector, electronic processing unit, and light source, eliminating the need for connection to surrounding pixels. Most importantly, the fast switching speed of these electronic components enables weight updates at speeds of several hundred megahertz (MHz), which is much faster than the several kHz speed of the spatial light modulator (LCD, DMD) described previously in Figures 28 to 34.

[0443] Each pixel's memory stores the weights or bias values ​​to be applied according to the program. Ultimately, a smart pixel light modulator can be thought of as an array of optical repeaters, each with its own memory.

[0444] Meanwhile, light rays passing through the smart pixel light modulators (3103a, 3103b) are input to the lens units (3104a, 3104b, 3105). The lens units (3104a, 3104b, 3105) are formed between a plurality of smart pixel light modulators (3103a, 3103b) and a plurality of first photodetectors (3120a, 3120b).

[0445] Additionally, the lens units (3104a, 3104b, 3105) serve to focus light emitted from different first light sources (3101a, 3101b) passing through pixels at the same relative positions of each smart pixel light modulator (3103a, 3103b) onto one first light detector (3120a, 3120b).

[0446] In an embodiment of the present invention, the lens units (3104a, 3104b, 3105) are formed corresponding to the plurality of smart pixel light modulators (3103a, 3103b), respectively, and include a plurality of second lenses (3104a, 3104b) that focus light rays to a single point, and a third lens (3105) that passes all light rays that have passed through the plurality of second lenses (3104a, 3104b). If the light rays incident on the second lenses (3104a, 3104b) are parallel light rays, they are focused at a focus.

[0447] The light passing through each smart pixel light modulator (3103a, 3103b) passes through each corresponding second lens (3104a, 3104b), is focused, and then spreads out again, where it passes through the third lens (3105) again.

[0448] When a ray passes through the second lens (3104a, 3104b), the angle of the ray changes depending on the height of the ray away from the optical axis of the second lens (3104a, 3104b). The ray (3110a) passing through the highest position on the optical axis of the second lens (3104a) and the ray (3110b) passing through the highest position on the optical axis of the second lens (3104b) have the same angle with respect to the horizontal axis.

[0449] The two light rays (3110a, 3110b) enter the third lens (3105) while forming parallel light, and thus converge at the same point on the focal plane of the third lens (3105). In Fig. 33, the two light rays converge on one first photodetector (3120a).

[0450] Similarly, light rays (3112a) passing through the second lens (3104a) and light rays (3112b) passing through the second lens (3104b), which are rays at relatively the same position, pass through the third lens (3105) and are then gathered on one photodetector (3120b).

[0451] The first photodetector (3120a, 3120b) can be composed of an optical semiconductor element, such as a photodiode. The current flowing in the first photodetector (3120a, 3120b) is ultimately the sum of the intensities of light arriving at each of the first photodetectors (3120a, 3120b). , It has a value proportional to the current value obtained from the first photodetector (3120a, 3120b) or its amplified value and the bias value ( stored in the electronic processing unit (3121a, 3121b) respectively , ) is added, and the sigmoid function б is calculated using that value as input, and the output value of the neural network node is obtained.

[0452] The rays (3111a, 3113a, 3111b, 3113b) show examples of rays traveling in the reverse direction. If the optical module operates in the forward direction, the rays from the first light source (3101a, 3101b) can also travel in the forward direction, in which case the pixels , , , By setting it to the OFF state, the light rays from the first light source (3101a, 3101b) can not pass through. In fact, the reverse light rays (3111a, 3113a, 3111b, 3113b) start from the first photodetector (3120a, 3120b) and reach the first light source (3101a, 3101b), so they do not generate a signal and are an example of the reverse light path.

[0453] In one embodiment, each output value of the electronic processing unit (3121a, 3121b) can be transmitted to the connected second light source (3122a, 3122b), respectively. Accordingly, each output value of the electronic processing unit (3121a, 3121b) , can be adjusted proportionally.

[0454] Each electronic processing unit (3121a, 3121b) can use other activation functions, such as the RELU function, in addition to the sigmoid function.

[0455] In addition to performing function operations, the electronic processing units (3121a, 3121b) can store result values ​​or external input values ​​for use in calculations. This memory function enables various calculations by comparing calculated values ​​with stored values ​​in the next sequence or by using stored values ​​for function calculations.

[0456] The electronic processing unit has a communication function and can exchange data with surrounding electronic processing units or external processors through connected communication lines.

[0457] The second light source (3122a, 3122b) is the neural network. , It corresponds to and is used as the input value for the next layer.

[0458] The electronic processing unit (3121a, 3121b) is an electronic integrated circuit formed on a semiconductor substrate and can be implemented through an analog circuit or a digital circuit.

[0459] The lenses used in this embodiment can be implemented using refractive lenses or diffractive optical elements.

[0460] In this way, the optoelectronic module (3010) has one output node, for example: The first photodetector (3120b) and the third light source (3124b) are connected through an electronic processing unit (3121b), so that light can be sent in the reverse direction from the third light source (3124b) to send a signal to the photodetector (3174) in the smart pixel light modulator (3103a).

[0461] The electronic processing unit (3173A) within the smart pixel can convert the current value of the photodetector and the weight information stored in the memory into a calculated current to control the intensity of the light source (3172) within the smart pixel. In this way, the optoelectronic module (3010) of FIG. 33 can implement the bidirectional optical neural network function of the optoelectronic module (3050) of FIG. 28 using the smart pixel optical modulator. Since the update speed of the smart pixel optical modulator is much faster than that of a conventional spatial light modulator, the parallel processing speed of the backpropagation algorithm, which requires rapid weight changes after forward calculation, can be significantly improved. This characteristic can contribute to increasing the learning speed of the artificial neural network.

[0462] FIG. 35 is a conceptual diagram of a bidirectional optoelectronic module using a smart pixel optical modulator including fan-in and fan-out according to a sixth embodiment of the present invention, and FIG. 36 is a conceptual diagram of the smart pixel optical modulator of FIG. 35.

[0463] The illustrated optoelectronic module (3020) includes a plurality of electrical input / outputs (3200), a plurality of smart pixel optical modulators (3201), lens units (3202, 3203), and a plurality of photodetectors (3204). In this way, the optoelectronic module (3020) is a structure that further simplifies the structure of FIG. 33, in which the first lens is eliminated and instead, an electrical fan-out, fan-in structure (3208, 3209) is included in the smart pixel optical modulator (3201).

[0464] Even if the distance between the second lens (3202) and the third lens (3203) is narrowed compared to FIG. 33, the pixel image of the smart pixel light modulator (3201) can be focused on the first photodetector (3204). In addition, the second lens (3202) and the third lens (3203) can be integrated into a single element, and a diffractive optical element (DOE) can perform this function. At this time, the more detailed structure of the third light source (3205) is as shown in FIG. 29, and multiple light rays can be sent in multiple directions using a diffraction grating (3450a) and a prism (3451a). The diffraction grating (3450a) and the prism (3451a) can be manufactured into a single optical element using a DOE. Additionally, a lens can be added between the diffraction grating (3450a) and the light source (3424a) to control the divergence angle of the light, and the DOE can implement this function in a single device.

[0465] A more detailed structure of the smart pixel optical modulator (3201) is shown in FIG. 36. The smart pixel includes both a light source (3212) and a photodetector (3211) on the second surface, and includes both a fan-out and fan-in structure connecting the electronic processing unit (3210) and the electrical input / output nodes.

[0466] The electrical fan-in structure calculates the signal acquired from the photodetector (3211) with the weights of the memory, converts the result into a current, and transmits it to the input / output node, and the electrical fan-out structure distributes the input signal into multiple branches through wires and transmits it. This structure enables the smart pixel optical modulator (3201) to operate in both directions and enables the optical neural network computer to compute algorithms such as the backpropagation algorithm. Since the smart pixel optical modulator (3201) has a fast switching speed and does not harm parallelism, it greatly improves the parallel processing capacity of the bidirectional optical neural network computer.

[0467] Figure 37 is a conceptual diagram of implementing input / output node scale-up in an optical neural network computer using a smart pixel optical modulator.

[0468] Figures 37(a) to 37(d) show a scale-up algorithm that doubles the number of input and output nodes in an optical neural network computer using a smart pixel optical modulator.

[0469] In Fig. 37(a), weight calculation is performed by fully connecting input nodes 1 to N and output nodes 1 to N. Then, output nodes 1 to N are stored in the memory of each pixel, and weight calculation is performed by fully connecting input nodes 1 to N and output nodes N+1 to 2N.

[0470] In the next step, as shown in Fig. 37(b), the input node values ​​from 1 to N are stored in each pixel memory, and the input node values ​​from N+1 to 2N are read from the memory. The fully connected weights are calculated between the input node values ​​from N+1 to 2N and the output node values ​​from N+1 to 2N. Then, in the output stage, the output node values ​​from N+1 to 2N are stored in each pixel memory, and the output node values ​​from 1 to N are read from the memory. The fully connected weights are calculated between the input node values ​​from N+1 to 2N and the output node values ​​from 1 to N, and then added to the previous value and stored in the memory.

[0471] Although these processes involve nine computational steps, they double the number of input and output nodes, quadrupling the number of connections. Furthermore, compared to conventional clustering methods, they eliminate the need for a significant increase in optical module clustering space or the need for manufacturing multiple optical modules, making them a highly economical solution.

[0472] In addition, since the number of input nodes or output nodes can be expanded to arbitrary multiples such as 2, 3, 4, and 5, it provides the advantage of being able to flexibly adjust the connecting topology without being limited by the hardware size. In the case of optical neural network computers using existing spatial light modulators (LCD, DMD), the switching speed is slow, so the weight update speed is slow, and when such connections are made, the parallel processing speed is greatly reduced, but the optical neural network computer using smart pixels does not have this drawback.

[0473] The optical computer proposed in the present invention can perform and apply various algorithms, such as neural network learning algorithms such as backpropagation algorithms or autocorrelation algorithms, as well as inference calculations, because light can travel in both directions.

[0474] Furthermore, the time delay in computation occurs in the electronic processing unit, but since it is performed in parallel across all outputs, it can be viewed as equivalent to one instruction cycle of a digital electronic circuit. When configuring L multilayer neural networks, the actual computation speed increases by NXMXL times because computations are performed simultaneously across all layers.

[0475] Furthermore, since bidirectional computation is possible, calculations can be performed by round-tripping between two neural network layers. This allows for computation of neural networks corresponding to any number of layers within a two-layer bidirectional optical neural network, significantly reducing hardware requirements. While conventional bidirectional optical neural networks have been consistently slow due to slow weight update rates, optical neural networks using smart pixel optical modulators not only save hardware space but also offer extremely fast parallel processing speeds. For a detailed explanation, please refer to Reference 1.

[0476] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single entity may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined manner.

[0477] The scope of the present invention is indicated by the claims described below, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

Claims

1. A plurality of first light sources formed at a certain distance from each other; A plurality of first lenses each formed to correspond to a plurality of first light sources and reducing the divergence angle of light emitted from the corresponding first light sources; A plurality of smart pixel light modulators each formed in correspondence with a plurality of first lenses and controlling the intensity of light passing through the corresponding first lens through each pixel having a weight stored in a memory; A plurality of photodetectors formed spaced apart from a plurality of smart pixel optical modulators and obtaining a current value according to the intensity of light; and An optoelectronic module using a smart pixel optical modulator, comprising: a lens unit formed between a plurality of smart pixel optical modulators and a plurality of optical detectors, the lens unit collecting light emitted from different first light sources passing through pixels at the same relative positions of each smart pixel optical modulator onto a single optical detector.

2. In paragraph 1, the lens part, A plurality of second lenses formed to correspond to each of the plurality of smart pixel light modulators and to focus light rays into one point; and An optoelectronic module using a smart pixel optical modulator, comprising a third lens for passing all light rays passing through the plurality of second lenses.

3. In paragraph 1, An optoelectronic module using a smart pixel optical modulator, wherein the smart pixel optical modulator is configured in a form in which a plurality of pixels are arranged, and each pixel is assigned a photodetector, an electronic processing unit, and a light source.

4. In paragraph 3, The above photodetector converts light passing through the first lens into a current signal, An optoelectronic module using a smart pixel optical modulator, wherein the electronic processing unit amplifies the current signal or converts the result of calculating the current signal with a weight value stored in a memory into a current signal and outputs the result to the light source.

5. Multiple electrical input nodes formed at a certain distance from each other; A plurality of smart pixel optical modulators, each composed of a plurality of pixels, each of which connects an input signal input through the electrical input node to each of the pixels through an electrical fan-out structure; A plurality of photodetectors formed spaced apart from the plurality of smart pixel optical modulators and obtaining a current value according to the intensity of light; and An optoelectronic module using a smart pixel optical modulator, comprising: a lens unit formed between the plurality of smart pixel optical modulators and the plurality of optical detectors, the lens unit collecting light rays originating from different smart pixel optical modulators passing through pixels at the same relative positions of each smart pixel optical modulator onto one optical detector.

6. In the fifth paragraph, the lens part, A plurality of second lenses formed to correspond to each of the plurality of smart pixel light modulators and to focus light rays into one point; and An optoelectronic module using a smart pixel optical modulator, comprising a third lens for passing all light rays passing through the plurality of second lenses.

7. In paragraph 5, An optoelectronic module using a smart pixel optical modulator, wherein the smart pixel optical modulator is configured in a form in which a plurality of pixels are arranged, and each pixel is assigned an electronic processing unit and a light source.

8. In paragraph 7, An optoelectronic module using a smart pixel optical modulator, wherein the input signal is connected to the electronic processing unit through the fan-out structure, and the electronic processing unit converts the result of calculating the input signal with a weight value stored in a memory into a current signal and outputs the result to the light source.

9. In paragraph 8, An optoelectronic module using a smart pixel optical modulator, wherein the intensity of the output light source of each pixel is proportional to the product of the input signal and the weight value stored in the memory.

10. In paragraph 6, An optoelectronic module using a smart pixel optical modulator, wherein the second and third lenses are bonded to form a single optical component to integrate their functions.

11. In paragraph 6, An optoelectronic module using a smart pixel optical modulator in which the second and third lenses are combined into a single lens using a diffractive optical element (DOE) to integrate functions.

12. An optical neural network computer comprising an optoelectronic module using a smart pixel optical modulator according to paragraph 1.

13. A plurality of first light sources formed at a certain distance from each other; A plurality of first lenses formed to correspond to each of the plurality of first light sources and reducing the divergence angle of light emitted from the corresponding first light sources; A plurality of smart pixel light modulators each formed corresponding to a plurality of first lenses and controlling the intensity of light passing through the corresponding first lens through each pixel having a weight stored in a memory; A plurality of first photodetectors formed spaced apart from the plurality of smart pixel optical modulators and obtaining a current value according to the intensity of light; and A lens unit formed between the plurality of smart pixel light modulators and the plurality of first light detectors, the lens unit collecting light emitted from different first light sources passing through pixels at different relative positions of each smart pixel light modulator onto one first light detector; The above lens part, A plurality of second lenses formed corresponding to each of the plurality of smart pixel optical modulators and concentrating light rays into one point; and An optoelectronic module using a smart pixel optical modulator, comprising a plurality of third lenses corresponding to each of the second lenses, wherein the third lenses pass light passing through the second lens.

14. In paragraph 13, Each smart pixel light modulator and the second lens corresponding to each smart pixel light modulator are formed to be spaced apart by a first distance, and each first light detector is formed to be spaced apart by a second distance from the third lens. An optoelectronic module using a smart pixel optical modulator, wherein the second distance satisfies a relationship in which the distance at which the image of the pixel of the smart pixel optical modulator is formed on the first photodetector does not deviate by more than a certain error value.

15. In paragraph 13, The above lens part, An optoelectronic module using a smart pixel optical modulator, which implements a function of calculating a convolution of a light source array and a pixel array of a smart pixel optical modulator by using a function of gathering light emitted from different first light sources passing through pixels at different relative positions of each smart pixel optical modulator into a single first optical detector, formed between the plurality of smart pixel optical modulators and the plurality of first optical detectors.

16. In paragraph 13, An optoelectronic module using a smart pixel optical modulator, wherein a plurality of photodetectors corresponding to the third lens or positioned within the third lens aperture are provided, and each photodetector performs a different convolution operation on light passing through a smart pixel optical modulator pixel at a different relative position.

17. In paragraph 13, An optoelectronic module using a smart pixel optical modulator, wherein the smart pixel optical modulator is configured in a form in which a plurality of pixels are arranged, and each pixel is assigned a photodetector, an electronic processing unit, and a light source.

18. In paragraph 17, The above photodetector converts light passing through the first lens into a current signal, An optoelectronic module using a smart pixel optical modulator, wherein the electronic processing unit amplifies the current signal or converts the result of calculating the current signal with a weight value stored in a memory into a current signal and outputs the result to the light source.

19. In paragraph 13, A plurality of second light sources formed at a predetermined distance from the plurality of first light detectors and from which light travels in the opposite direction to the first light source; and An optoelectronic module using a smart pixel optical modulator, further comprising a plurality of second optical detectors formed at a predetermined distance from the first light source, collecting light rays irradiated from the plurality of second light sources and passing through the plurality of smart pixel optical modulators and the first lens.

20. Multiple electrical input / output nodes formed at a certain distance from each other; A plurality of smart pixel optical modulators, each composed of a plurality of pixels, each of which connects an input signal input through the electrical input node to each of the pixels through an electrical fan-out structure; A plurality of first photodetectors formed spaced apart from the plurality of smart pixel optical modulators and obtaining a current value according to the intensity of light; and An optoelectronic module using a smart pixel optical modulator, comprising: a lens formed between the plurality of smart pixel optical modulators and the plurality of first optical detectors, the lens collecting light rays originating from different smart pixel optical modulators passing through pixels at different relative positions of each smart pixel optical modulator onto one optical detector.

21. In paragraph 20, Each smart pixel light modulator and each corresponding lens to each smart pixel light modulator are formed to be spaced apart by a first distance, and each first photodetector is formed to be spaced apart by a second distance from the lens. An optoelectronic module using a smart pixel optical modulator, wherein the second distance satisfies a relationship in which the distance at which the image of the pixel of the smart pixel optical modulator is formed on the first photodetector does not deviate by more than a certain error value.

22. In paragraph 20, The above lens, An optoelectronic module using a smart pixel optical modulator, which implements a function of calculating a convolution of a light source array and a pixel array of a smart pixel optical modulator by using a function of gathering light emitted from different first light sources passing through pixels at different relative positions of each smart pixel optical modulator into a single first optical detector, formed between the plurality of smart pixel optical modulators and the plurality of first optical detectors.

23. In paragraph 20, An optoelectronic module using a smart pixel optical modulator, wherein a plurality of photodetectors corresponding to the above lens or arranged within the lens aperture are provided, and each photodetector performs a different convolution operation on light passing through pixels of the smart pixel optical modulator at different relative positions.

24. In paragraph 20, An optoelectronic module using a smart pixel optical modulator, wherein the smart pixel optical modulator is configured in a form in which a plurality of pixels are arranged, and each pixel is assigned a photodetector, an electronic processing unit, and a light source.

25. In paragraph 24, The above photodetector converts light passing through the lens into a current signal, An optoelectronic module using a smart pixel optical modulator, wherein the electronic processing unit amplifies the current signal or converts the result of calculating the current signal with a weight value stored in a memory into a current signal and outputs the result to the light source.

26. In paragraph 20, It further includes a plurality of light sources formed at a predetermined distance from the plurality of first photodetectors and having light traveling in the opposite direction to the input signal; An optoelectronic module using a smart pixel optical modulator, wherein the smart pixel optical modulator comprises a plurality of photodetectors that collect light rays irradiated from the plurality of light sources and passed through the lens, an electronic processing unit connected to the photodetectors, and an electrical fan-in structure that connects the electrical input / output node and the plurality of pixels.

27. An optical convolutional neural network computer comprising an optoelectronic module using a smart pixel optical modulator according to Article 13.

28. A plurality of first light sources formed at a certain distance from each other; A plurality of first lenses formed to correspond to each of the plurality of first light sources and reducing the divergence angle of light emitted from the corresponding first light sources; A plurality of smart pixel light modulators each formed corresponding to the plurality of first lenses and controlling the intensity of light passing through the corresponding first lens through each pixel having a weight stored in a memory; A plurality of first photodetectors formed spaced apart from the plurality of smart pixel optical modulators and obtaining a current value according to the intensity of light; A lens unit formed between the plurality of smart pixel light modulators and the plurality of first light detectors, the lens unit collecting light emitted from different first light sources passing through pixels at the same relative positions of each smart pixel light modulator onto one first light detector among the plurality of first light detectors; A plurality of third light sources formed at a certain distance from the plurality of first light detectors and having light traveling in the opposite direction to the first light source; and A bidirectional optoelectronic module using a smart pixel optical modulator, comprising: a plurality of third optical detectors formed at a predetermined distance from the first light source, collecting light rays irradiated from the plurality of third light sources and passing through the smart pixel optical modulator and the first lens; 29. In paragraph 28, A bidirectional optoelectronic module using a smart pixel optical modulator, wherein the plurality of first photodetectors are formed on the same substrate as the plurality of third light sources, and the plurality of third photodetectors are formed on the same substrate as the plurality of first light sources.

30. In paragraph 28, The above lens part, A plurality of second lenses formed corresponding to each of the plurality of smart pixel light modulators and concentrating light rays irradiated from the first light source into one point; and A bidirectional optoelectronic module using a smart pixel optical modulator, comprising a third lens for passing all light rays passing through the plurality of second lenses.

31. In paragraph 28, A bidirectional optoelectronic module using a smart pixel optical modulator, wherein the plurality of third light sources are configured to include at least one of a combination of a diffraction grating and a prism or a diffractive optical element (DOE).

32. In paragraph 28, A bidirectional optoelectronic module using a smart pixel optical modulator, wherein the smart pixel optical modulator is configured in a form in which a plurality of pixels are arranged, and each pixel is assigned a photodetector, an electronic processing unit, and a light source.

33. In paragraph 32, The above photodetector converts light passing through the first lens into a current signal, A bidirectional optoelectronic module using a smart pixel optical modulator, wherein the electronic processing unit amplifies the current signal or converts the result of calculating the current signal with a weight value stored in a memory into a current signal and outputs the result to the light source.

34. Multiple electrical input / output nodes formed at a certain distance from each other; A plurality of smart pixel optical modulators, each composed of a plurality of pixels, each of which connects an input signal input through the electrical input / output node to each of the pixels through an electrical fan-out structure; A plurality of photodetectors formed spaced apart from the plurality of smart pixel optical modulators and obtaining a current value according to the intensity of light; A lens unit formed between the plurality of smart pixel optical modulators and the plurality of optical detectors, the lens unit collecting light rays from different smart pixel optical modulators that depart from pixels at the same relative positions of each smart pixel optical modulator onto one optical detector; and A plurality of light sources formed at a certain distance from the plurality of photodetectors and having light traveling in the opposite direction to the input signal; A bidirectional optoelectronic module using a smart pixel optical modulator, wherein the smart pixel optical modulator comprises a plurality of photodetectors that collect light rays irradiated from the plurality of light sources and passing through the lens unit, an electronic processing unit connected to the photodetectors, and an electrical fan-in structure that connects the electrical input / output node and the plurality of pixels.

35. In paragraph 34, the lens part, A plurality of second lenses formed to correspond to each of the plurality of smart pixel light modulators and to focus light rays into one point; and A bidirectional optoelectronic module using a smart pixel optical modulator, comprising a third lens for passing all light rays passing through the plurality of second lenses.

36. In paragraph 34, A bidirectional optoelectronic module using a smart pixel optical modulator, wherein the smart pixel optical modulator is configured in a form in which a plurality of pixels are arranged, and each pixel is assigned a photodetector, an electronic processing unit, and a light source.

37. In paragraph 36, A bidirectional optoelectronic module using a smart pixel optical modulator, wherein the input signal is connected to the electronic processing unit through the fan-out structure, and the electronic processing unit converts the result of calculating the input signal with a weight value stored in a memory into a current signal and outputs the result to the light source.

38. In paragraph 35, A bidirectional optoelectronic module using a smart pixel optical modulator, wherein the second and third lenses are bonded to form a single optical component and integrate their functions.

39. In paragraph 35, A bidirectional optoelectronic module using a smart pixel optical modulator, wherein the second and third lenses are combined into a single lens using a diffractive optical element (DOE) to integrate functions.

40. An optical neural network computer comprising a bidirectional optoelectronic module using a smart pixel optical modulator according to Article 28.

Images