Bidirectional optoelectronic module using spatial light modulator and optical neural network computer comprising same

Abstract

The present invention relates to a bidirectional optoelectronic module using a spatial light modulator, comprising: a plurality of first light sources; a plurality of first lenses for reducing a divergence angle of light rays emitted from the first light sources; a plurality of spatial light modulators for adjusting the intensity of light rays having passed through the first lenses; a plurality of first photodetectors for acquiring current values according to the intensity of light; a lens unit for collecting light rays emitted from different first light sources to one of the plurality of first photodetectors; a plurality of third light sources through which light travels in a direction opposite to that of the first light sources; and a plurality of third photodetectors that collect light rays which have been emitted from the plurality of third light sources and passed through the spatial light modulators and the first lenses, and are formed to be spaced apart from the first light sources by a predetermined distance.

Description

Bidirectional optoelectronic module using spatial light modulator and optical neural network computer including the same

[0001] The present invention relates to a bidirectional optoelectronic module using a spatial light modulator and an optical computer including the same, and more particularly, to a configuration of a high-speed optoelectronic module and an optical neural network computer that operate in parallel by combining a light source, a lens, a spatial light modulator, a photodetector, etc. instead of a complex electronic circuit.

[0002] Recently, in fields such as image processing, the demand for high-speed computational processing technology targeting two-dimensional and three-dimensional information has been rapidly increasing.

[0003] An optical computer is a computer that uses an optical integrated circuit (IC) that utilizes the characteristics of light in its computational circuitry, allowing for more redundant communication than with a telecommunication line.

[0004] Current electronic calculators are comprised of electronic circuits. Consequently, computational speeds are limited by factors such as component delays and stray capacitance, limiting their potential for high-speed computing. Optical computers are expected to overcome these limitations.

[0005] Meanwhile, when implementing an analog neural network electrically and simultaneously performing calculations, problems arise, such as electromagnetic noise due to overlapping wires and complex circuit wiring layout. To address these issues, a previous patent (Patent Document 4) proposed an optoelectronic module using a spatial light modulator and an optical computer including the same.

[0006] The optical computer architecture presented in patent document 4 enables parallel calculations corresponding to the number of pixels of a spatial light modulator compared to an electronic computer, thereby increasing the processing capacity by 10 when using a Full HD LCD microdisplay. 6This has the effect of increasing the resolution by about 10 times, and when using a micro-optical system such as a micro-lens array, it offers the advantage of miniaturizing the system to a size of about 10 mm × 10 mm × 10 mm. However, one drawback is that the direction of light propagation is fixed from the first substrate to the second substrate, making it difficult to move data in the opposite direction.

[0007] In neural network calculations, when weights are determined, data starts at the input layer and moves in one direction to the next layer, so existing optical computer structures can process data at a very high speed, which is not a problem. This process is called inference, and it can be viewed as a case where the neural network weights have already been determined and image processing is performed on various input images. However, in the case of training calculations that require finding the weights themselves, a backpropagation algorithm must be used, but it is not easy to perform this backpropagation algorithm with the structure of patent document 4.

[0008] [Prior Art Literature]

[0009] (Patent Document 1) KR 10-0624852 B1

[0010] (Patent Document 2) KR 10-2079833 B1

[0011] (Patent Document 3) JP 6746074 B2

[0012] (Patent Document 4) KR 10-2622313 B1

[0013] The present invention has been devised to solve the above problems, and an object of the present invention is to provide a bidirectional optoelectronic module using a spatial light modulator and an optical neural network computer including the same.

[0014] Another object of the present invention is to provide an optoelectronic module using a spatial light modulator capable of bidirectional optical connection between substrates of optical neural network hardware, and an optical neural network computer including the same.

[0015] In order to achieve the above object, according to one embodiment of the present invention, a bidirectional optoelectronic module using a spatial light modulator comprises: a plurality of first light sources formed at a predetermined distance from each other; a plurality of first lenses formed corresponding to the plurality of first light sources, respectively, and reducing a divergence angle of light emitted from the corresponding first light sources; a plurality of spatial light modulators formed corresponding to the plurality of first lenses, respectively, and adjusting the intensity of light passing through the corresponding first lens through respective pixels having preset weights; a plurality of first photodetectors formed spaced apart from the plurality of spatial light modulators, each of which acquires a current value according to the intensity of light; a lens unit formed between the plurality of spatial light modulators and the plurality of first photodetectors, each of which collects light emitted from different first light sources passing through pixels at the same relative positions of the respective spatial light modulators onto one of the plurality of first photodetectors; a plurality of third light sources formed at a predetermined distance from the plurality of first photodetectors, the light of which travels in a direction opposite to the first light sources; And a plurality of third light detectors formed at a certain distance from the first light source, collecting light rays irradiated from the plurality of third light sources and passing through the spatial light modulator and the first lens.

[0016] According to one aspect of the present invention described above, by providing an optoelectronic module using a spatial light modulator and an optical neural network computer including the same, an analog / digital optical computer structure using optical connections instead of purely electronic circuits is implemented, so that even if the number of input and output terminals increases to N and M, respectively, the connections do not become complicated or noise does not occur as in electronic circuits. Accordingly, since important part calculations are performed in parallel by performing approximately NXM multiplications and additions at once, ultra-small and ultra-high-speed calculations are possible.

[0017] In addition, compared to the existing optical computer structure that made it difficult to perform algorithms such as reverse propagation because light propagated in one direction, the present invention enables light to propagate in both directions, enabling parallel high-speed processing of various algorithms such as the reverse propagation algorithm required for neural network learning.

[0018] Figure 1 is a conceptual diagram explaining the conventional backpropagation algorithm in a neural network.

[0019] Figure 2 is a conceptual diagram of a non-differential mode bidirectional optoelectronic module using a spatial light modulator according to one embodiment of the present invention.

[0020] Figure 3 is a conceptual diagram of the third light source of Figure 2 implemented using a diffraction grating and a prism.

[0021] Fig. 4 is a drawing showing an example of cascading the optoelectronic modules of Fig. 2;

[0022] FIG. 5 is a conceptual diagram of a differential mode bidirectional optoelectronic module using a spatial light modulator according to another embodiment of the present invention.

[0023] And, Fig. 6 is a drawing showing an example of cascading the optoelectronic modules of Fig. 5.

[0024] The following detailed description of the present invention refers to the accompanying drawings, which illustrate specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention, while different from each other, are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the present invention. Furthermore, it should be understood that the positions or arrangements of individual components within each disclosed embodiment may be modified without departing from the spirit and scope of the present invention. Accordingly, the following detailed description is not intended to be limiting, and the scope of the present invention is defined only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly described. Like reference numerals in the drawings designate the same or similar functionality throughout the several aspects.

[0025] The components according to the present invention are defined by functional distinctions rather than physical distinctions, and can be defined by the functions each component performs. Each component may be implemented as hardware or program code and processing units that perform each function, and the functions of two or more components may be implemented by including them in a single component. Therefore, the names given to the components in the following embodiments are not intended to physically distinguish each component, but rather to suggest the representative functions performed by each component, and it should be noted that the technical spirit of the present invention is not limited by the names of the components.

[0026] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

[0027] Figure 1 is a conceptual diagram explaining the backpropagation algorithm in an artificial neural network. Weighted input This weight class It is used between, and this feature theoretically has the advantage that the development of the theory is easier when using weighted input. The first layer ( ) node value is the weight Multiply by and then bias Add weighted input and construct a sigmoid function Enter the output value of the next layer node This is shown in the following mathematical expressions 1 and 2.

[0028]

[0029]

[0030] Assuming that the second layer is the last layer of the neural network, the target value of each node during learning is Then, the sum of the squares of the differences between the output value of the node and the target value can be defined as the error or merit function E as in mathematical expression 3.

[0031]

[0032] The principle of learning in artificial neural networks is the process of adjusting weights and biases or weighted inputs to minimize the error value. In this process, a correction value is added to the weights or weighted inputs. The steepest gradient method, which is one of the minimization algorithms, is used, which views the weights and weighted inputs as variables, calculates the gradient vector for these variables, changes the sign, and then adds the correction value in that direction. In artificial intelligence theory, this gradient vector is also called a sensitivity vector, and here, is expressed as . In addition, this sensitivity is proportional to the weights, bias, error, or correction amount of the weighted input. The sensitivity vector for the weighted input of the first layer is shown in the following mathematical expression 4.

[0033]

[0034] By applying the chain rule of partial differentiation and equations 1 to 3, If we obtain , we can obtain the following mathematical equation 5. The final output error is multiplied by the function that differentiates the sigmoid function.

[0035]

[0036] The relationship between the sensitivity of the first layer and the sensitivity of the second layer can also be obtained by applying the chain rule and equations 1 and 2. The result is shown in equation 6 below. Adjacent weighted inputs are connected by weights and a sigmoid differential function. Ultimately, this structure, in the form of a recurrence relation, allows us to derive the weighted input of the first layer by knowing the weighted input of the second layer. Equation 5 provides the initial value of the recurrence relation.

[0037]

[0038] Similarly, the sensitivity of the weights can be calculated using the chain rule and Equation 2, as shown in Equation 7 below. The sensitivity of the weights is equal to the product of the sensitivity of the weighted inputs of the subsequent layer and the node values ​​of the previous layer. Using the weighted input sensitivity vector of each layer, obtained using the recurrence relation, the sensitivity of the weights can also be calculated.

[0039]

[0040] The bias sensitivity can also be obtained in the same way, and is equal to the value of the weight sensitivity assuming that the input node value is 1. The result is shown in Equation 8 below.

[0041]

[0042] In this way, the weight and bias sensitivity vectors of each layer can be obtained, and the error function can be minimized by continuously modifying them by changing the sign of the sensitivity, multiplying it by a certain coefficient, and adding it to the previous value. This backpropagation algorithm is a general ( )th layer, ( ) th layer, final When applied to the th layer, the following mathematical expressions 9 to 13 can be obtained.

[0043]

[0044]

[0045]

[0046]

[0047]

[0048] As shown in Equation 11, the weighted input sensitivity of the preceding layer is the weighted value added to the sensitivity of the subsequent layer, which is basically similar to the forward artificial neural network calculation except that the direction of propagation is changed. Therefore, the lens array or spatial light modulator array used in existing optical neural network computers can be used similarly. However, sending light in both the forward and reverse directions requires additional light source arrangements, detector arrangements, and securing an optical path for reverse propagation, which necessitates a new system structure, such as a structural change in the light source.

[0049] FIG. 2 is a conceptual diagram of a non-differential mode bidirectional optoelectronic module using a spatial light modulator according to one embodiment of the present invention.

[0050] An optoelectronic module (50, hereinafter referred to as an optoelectronic module) using a spatial light modulator according to the present invention operates in parallel by combining a light source, a lens, a spatial light modulator, a light detector, etc. instead of a complex electronic circuit. The optoelectronic module (50) according to the present invention can be applied to an optical neural network computer.

[0051] Referring to FIG. 2, an optical optoelectronic module (50) according to one embodiment of the present invention includes a plurality of first light sources (401a, 401b), a plurality of first lenses (402a, 402b), a plurality of spatial light modulators (403a, 403b), a plurality of first light detectors (420a, 420b), a plurality of third light sources (424a, 424b), a plurality of third light detectors (440a, 440b), and a lens unit (404a, 404b, 405).

[0052] A plurality of first light sources (401a, 401b) may be formed at regular intervals on the first substrate (400), but may also be formed on different substrates.

[0053] An optical optoelectronic module (50) according to one embodiment of the present invention may further include a plurality of electronic processing units (421a, 421b, 441a, 441b), a plurality of second light sources (422a, 422b), and a plurality of second light detectors (460a, 460b).

[0054] A plurality of first and second photodetectors (420a, 420b, 460a, 460b), a plurality of electronic processing units (421a, 421b), a plurality of second light sources (422a, 422b), and a plurality of third light sources (424a, 424b) may be formed on a second substrate (430), but may alternatively be formed separately on another substrate or one or more elements may be formed on the same substrate. For example, the first substrate (400) and the second substrate (430) may be semiconductor substrates or PCBs (Printed Circuit Boards).

[0055] A plurality of first lenses (402a, 402b) can be formed on the same substrate.

[0056] Multiple spatial light modulators (403a, 403b) can be formed on the same substrate.

[0057] A plurality of second lenses (404a, 404b) can be formed on the same substrate.

[0058] A plurality of first light sources (401a, 401b) and third light detectors (440a, 440b) can be formed on the same substrate.

[0059] A plurality of third light sources (424a, 424b) and first light detectors (420a, 420b) can be formed on the same substrate.

[0060] In FIG. 2, the first light source (401a, 401b), the first lens (402a, 402b), the spatial light modulator (403a, 403b), the first photodetector (420a, 420b), the third light source (424a, 424b), the third photodetector (440a, 440b), the second light source (422a, 422b), the second photodetector (460a, 460b), etc. are expressed as two, and the electronic processing unit (421a, 421b, 441a, 441b) is expressed as four, but these are only examples, and the number of each element can be changed and designed as needed.

[0061] For example, in the present invention, the light source may be a light emitting diode (LED) or a semiconductor laser, particularly a vertical-cavity surface-emitting laser. The intensity of light can be controlled by varying the intensity of the current flowing through the light source, thereby individually controlling the input values.

[0062] Light emitted from a plurality of first light sources (401a, 401b) passes through a plurality of corresponding first lenses (402a, 402b), and then passes through a corresponding spatial light modulator (403a, 403b). The first lenses (402a, 402b) reduce the divergence angle of the light emitted from the corresponding first light sources (401a, 401b), and preferably modulate the light into parallel light.

[0063] In Fig. 2, four representative rays (410a, 412a, 410b, 412b) from among the light emitted from each first light source (401a, 401b) are shown, and the four rays pass through individual pixels in the spatial light modulator (403a, 403b). The spatial light modulator (403a, 403b) is formed corresponding to each of the plurality of first lenses (402a, 402b), and adjusts the intensity of the rays passing through the corresponding first lenses (402a, 402b) through each pixel having a preset weight.

[0064] For example, liquid crystal display panels (LCDs) can be used as each of the spatial light modulators (403a, 403b). Since the LCD panel regulates the intensity of incoming light to different transmittances according to voltage and passes the light through, the intensity of each light beam can be adjusted to a predetermined value.

[0065] Therefore, even if the light originates from the same light source, the intensity of the light passing through each pixel on the LCD panel varies depending on the transmittance of each pixel. For example, if the illuminance of the light source is 1.0 W / m 2 In this case, if the transmittance of the LCD pixel is 0.10, the illuminance of the output light is 0.10 W / m 2 And if the transmittance is 0.05, the illuminance of the output light is 0.05 W / m 2 It becomes.

[0066] The transmittance of the LCD panel can be adjusted in more than 256 steps depending on the voltage, so in this way, the product of the weights and input values ​​in the neural network can be implemented. Here, the input value is the intensity of the light source and the transmittance of the spatial light modulator corresponds to the weight.

[0067] Light rays passing through the spatial light modulators (403a, 403b) are input to the lens units (404a, 404b, 405). The lens units (404a, 404b, 405) are formed between a plurality of spatial light modulators (403a, 403b) and a plurality of first photodetectors (420a, 420b).

[0068] Additionally, the lens units (404a, 404b, 405) serve to collect light rays emitted from different first light sources (401a, 401b) passing through pixels at the same relative positions of each spatial light modulator (403a, 403b) onto one first light detector (420a, 420b).

[0069] In an embodiment of the present invention, the lens units (404a, 404b, 405) are formed corresponding to the plurality of spatial light modulators (403a, 403b), respectively, and include a plurality of second lenses (404a, 404b) that focus light rays to a single point, and a third lens (405) that passes all light rays that have passed through the plurality of second lenses (404a, 404b). If the light rays incident on the second lenses (404a, 404b) are parallel light rays, they are focused at a focal point.

[0070] The light rays passing through each spatial light modulator (403a, 403b) pass through the corresponding second lens (404a, 404b), are focused, and then spread out again, where they pass through the third lens (405) again.

[0071] When a light ray passes through the second lens (404a, 404b), the angle of the light ray changes depending on the height of the light ray away from the optical axis of the second lens (404a, 404b). The light ray (410a) passing through the highest position on the optical axis of the second lens (404a) and the light ray (410b) passing through the highest position on the optical axis of the second lens (404b) have the same angle with respect to the horizontal axis.

[0072] The two light rays (410a, 410b) enter the third lens (405) while forming parallel light, and thus converge at the same point on the focal plane of the third lens (405). In Fig. 2, the two light rays converge on one first photodetector (420a).

[0073] Similarly, light rays (412a) passing through the second lens (404a) and light rays (412b) passing through the second lens (404b), which are rays at relatively the same position, pass through the third lens (405) and are then gathered on one photodetector (420b).

[0074] The first photodetector (420a, 420b) can be composed of an optical semiconductor element, such as a photodiode. The current flowing in the first photodetector (420a, 420b) is ultimately the sum of the intensities of light arriving at each of the first photodetectors (420a, 420b). , It has a value proportional to the current value obtained from the first photodetector (420a, 420b) or its amplified value and the bias value ( stored in the electronic processing unit (421a, 421b) respectively. , ) is added, and the sigmoid function б is calculated using the value as input, and the output value of the neural network node is obtained.

[0075] The rays (411a, 413a, 411b, 413b) show examples of rays traveling in the reverse direction. When the optical module operates in the forward direction, the rays from the first light source (401a, 401b) can also enter and travel in the forward direction, and in this case, the pixels , , , By setting it to the OFF state, the light rays from the first light source (401a, 401b) can not pass through. In fact, the reverse light rays (411a, 413a, 411b, 413b) start from the first photodetector (420a, 420b) and reach the first light source (401a, 401b), so they do not generate a signal and are an example of the reverse light path.

[0076] In one embodiment, each output value of the electronic processing unit (421a, 421b) can be transmitted to the connected second light source (422a, 422b), respectively. Accordingly, each output value of the electronic processing unit (421a, 421b) , can be adjusted proportionally.

[0077] Each electronic processing unit (421a, 421b) can use other activation functions, such as the RELU function, in addition to the sigmoid function.

[0078] In addition to performing function operations, the electronic processing units (421a, 421b) can store result values ​​or external input values ​​for use in calculations. This memory function enables various calculations to be performed by comparing calculated values ​​in the next sequence with stored values ​​or by using stored values ​​for function calculations.

[0079] The electronic processing unit has a communication function and can exchange data with surrounding electronic processing units or external processors through connected communication lines.

[0080] The second light source (422a, 422b) is the neural network , It corresponds to and is used as the input value for the next layer.

[0081] The electronic processing unit (421a, 421b) is an electronic integrated circuit formed on a semiconductor substrate and can be implemented through an analog circuit or a digital circuit.

[0082] The lenses used in this embodiment can be implemented using refractive lenses or diffractive optical elements.

[0083] The first light source (401a, 401b) and the first lens (402a, 402b) corresponding thereto can be formed to be spaced apart by the focal length of the first lens (402a, 402b) within an error of 20%.

[0084] In addition, the spatial light modulators (403a, 403b) and the second lenses (404a, 404b) corresponding thereto, respectively, may be formed to be spaced apart from each other by the focal length of the second lenses (404a, 404b) within an error of 20%, and the first photodetectors (420a, 420b) may be formed to be spaced apart from the third lens (405) by the focal length of the third lens (405) within an error of 40%.

[0085] In particular, when the spatial light modulator (403a, 403b) is spaced apart from the second lens (404a, 404b) by the focal length of the second lens (404a, 404b), and the lens (405) is spaced apart from the photodetector (420a, 420b) by the focal length of the third lens (405), the image of the spatial light modulator (403a, 403b) is generated on the first photodetector, so that cross-talk can be reduced.

[0086] However, even if the distance between the spatial light modulator (403a, 403b) and the second lens (404a, 404b) is not exactly the focal length but slightly off, the distance between the third lens (405) and the first photodetector (420a, 420b) can be adjusted so that the image of the pixel of the spatial light modulator (403a, 403b) is formed on the first photodetector (420a, 420b), so that a certain degree of tolerance can be achieved.

[0087] Likewise, the distance between the first light source (401a, 401b) and the corresponding first lens (402a, 402b) can be adjusted to maintain performance even if the distance between other lenses or elements deviates slightly from the focal length.

[0088] For example, a plurality of first lenses (402a, 402b) corresponding to a plurality of light sources can be formed on a single substrate, a plurality of spatial light modulators (403a, 403b) can also be formed on a single substrate, and a plurality of second lenses (404a, 404b) can also be formed on a single substrate. This structure can facilitate the volume and optical alignment of the entire system.

[0089] In the above description, the forward operation of the artificial neural network optical module, in which light originates from the first light source (401a, 401b) of the first substrate (400) and arrives at the first photodetector (420a, 420b) on the second substrate (430), has been described. For reverse light propagation and data processing, an optical module structure is required in which light originates from the third light source (424a, 424b) on the second substrate (430) and arrives at the third photodetector (440a, 440b) on the first substrate (400).

[0090] Figure 2 shows rays (415a, 415b, 416a, 416b) originating from a third light source (424a, 424b). Rays (411a, 413a, 411b, 413b), indicated by thin lines, take a similar path to forward rays (410a, 412a, 410b, 412b) and correspond to reverse rays, but are used here as auxiliary lines to explain reverse main rays (415a, 415b, 416a, 416b). If the light beam originating from the actual third light source (424a, 424b) proceeds to the auxiliary line (411a, 413a, 411b, 413b), it will reach the first light source (401a, 401b) and thus cannot generate a signal from the third light detector (440a, 440b). However, if the main rays (415a, 415b, 416a, 416b) start in an upward direction compared to the auxiliary rays (411a, 413a, 411b, 413b), the two rays meet at an angle that is not parallel to the lens optical axis at the spatial light modulator (403a, 403b), and the main rays (415a, 415b, 416a, 416b) arrive at the third photodetector (440a, 440b) located below the first light source (401a, 401b). The reason why the two rays (411a, 415a) meet at the spatial light modulator is because the spatial light modulator (403a, 403b) and the third light source (424a, 424b) are in a distance relationship with the object.

[0091] When the main light beam (415a, 415b, 416a, 416b) starts in a downward direction compared to the auxiliary light beam (411a, 413a, 411b, 413b), it passes through the upper part of the first light source (401a, 401b) on the side where the first substrate (400) is located, and the third light detector (440a, 440b) must be located above the first light source (401a, 401b) to detect the light. In the case of the electronic processing unit (421a), the node value , , Because it is calculating or storing can be obtained. The bias sensitivity or correction value is given by Equation 9. can be obtained through. The third light source (424a) is connected to the electronic processing unit (421a) and can send light intensity proportional to , where the main light beam (415a) is directed to the pixel of the spatial light modulator (403a). and the intensity of the light passes through is proportional to . Here and The two pixels corresponding to are spatially separated but set to the same value.

[0092] Likewise, the main ray (416a) originating from the third light source (424b) has light proportional to the pixel of the spatial light modulator (403a) and the intensity of light passes through is proportional to. These two rays (415a, 416a) meet and are added at the third photodetector (440a). is converted into an electrical signal proportional to the electronic processing unit (441a). Multiply by can be obtained in the same way. can be obtained.

[0093] In addition, near each pixel of the spatial light modulator (403a), a photodetector and an electronic processing unit are preferably formed within the same substrate and connected to each pixel to measure the intensity of light passing through, store data, exchange information with surrounding pixels, and perform mathematical operations. and In an electronic processing unit close to Wow Weight sensitivity because it can be obtained can be obtained in the same way. , , can also be obtained. Ultimately, the optoelectronic module (50) can perform artificial neural network calculations not only in the forward direction but also in the reverse direction, and is capable of large-scale parallel calculations.

[0094] Figure 3 is a conceptual diagram of the third light source of Figure 2 implemented using a diffraction grating and a prism.

[0095] Referring to FIG. 3, a light beam starting from a third light source (424b) is separated into several diffracted light beams as it passes through a diffraction grating (450a), and its direction is bent as it passes through a prism (451a). The angle at which the light beams are bent is determined according to the inclination angle of the prism (451a), and the angle between the diffracted light beams is determined according to the period of the diffraction grating (450a). The third light source (424a) can be a semiconductor light source such as an LED (Light Emitting Diode) or a VCSEL (Vertical Cavity Surface Emitting Laser), and a lens can be added between the third light source (424a) and the diffraction grating (450a) to adjust the divergence angle. The diffraction grating (450a), the prism (451a), and the lens can be replaced with a diffractive optical element.

[0096] Fig. 4 is a drawing showing an example of cascading the optoelectronic modules of Fig. 2.

[0097] Referring to FIG. 4, the first optical computation layer (801) is repeated and appears in the second optical computation layer (802), and an arbitrary number of neural network layers can be implemented. The output value from the first optical computation layer (801), which is the preceding layer, can be connected to the input of the second optical computation layer (802), which is the succeeding layer.

[0098] Another method of constructing a multilayer neural network in FIG. 2 is to perform multilayer neural network calculations by having light travel back and forth between the first substrate (400) and the second substrate (430) and storing the calculation results in electronic processing units (421a, 421b, 441a, 441b). In this case, each time the light propagates in one direction, the weight values ​​of the spatial light modulators (403a, 403b) must be changed according to the values ​​of each layer. In this case, the weight pixels are updated by retrieving the values ​​stored in the memory of the electronic processing unit connected to the pixels.

[0099] FIG. 5 is a conceptual diagram of a differential mode bidirectional optoelectronic module using a spatial light modulator according to another embodiment of the present invention.

[0100] The optoelectronic module (70) using the spatial light modulator according to the present embodiment can be constructed in substantially the same configuration as the optoelectronic module (50) of FIG. 2, except for the configuration of the light source and the light detector for implementing differential mode calculation. Therefore, a repeated description of the same components as the optoelectronic module (50) of FIG. 2 is omitted.

[0101] Referring to FIG. 5, an optical optoelectronic module (70) according to one embodiment of the present invention includes a plurality of first A photodetectors (610a, 610c), a plurality of first B photodetectors (610b, 610d), a plurality of third light sources (620a, 620b, 620c, 620d), a plurality of third A photodetectors (540a, 540c), and a plurality of third B photodetectors (540b, 540d). In addition, a plurality of electronic processing units (630a, 630b, 550a, 550b) and corresponding second light sources (701a, 701b) may be further included.

[0102] The output values ​​of the first A photodetector (610a) and the first B photodetector (610b) can be connected to an electronic processing unit (630a), and the output values ​​of the first A photodetector (610c) and the first B photodetector (610d) can be connected to an electronic processing unit (630b).

[0103] The structure of this embodiment provides the advantage that the electronic processing unit (630a, 630b) can perform various operations using the values ​​of the two photodetectors. The easiest example is subtraction. For example, Let us assume a case of calculation. Here, superscripts are omitted to avoid the complexity of representing layers in the weight expression. Therefore, Is It represents.

[0104] Subtraction using light is possible using principles like destructive interference, but this increases the complexity of optical computing circuits, as it requires adjusting for path differences. Subtraction is not easy when two light beams enter a photodetector that only detects light intensity.

[0105] Therefore, in this embodiment, light corresponding to the (+) term is collected by the first A photodetector (610a, 610c), light corresponding to the (-) term is collected by the first B photodetector (610b, 610d), and the electronic processing unit (630a, 630b) calculates by subtracting the two signal intensities.

[0106] In the weight expression in Fig. 5 and Is and Represents the positive part of the weight, and Is and Represents the negative part of the weight. For example, is positive If is negative, , , , Using You can get results.

[0107] Even if there are any number of terms, the rays corresponding to the (+) terms are weighted Set the original model value, and the rays corresponding to the (-) term have weights After setting to 0, only the rays corresponding to the (+) term can be collected and added to the photodetector (610a). Similarly, the rays corresponding to the (-) term can be collected and sent to the photodetector (610b) for addition. Therefore, in this way, general neural network operations including negative weights can be performed.

[0108] Each electronic processing unit (630a, 630b) can use the output of not only two surrounding photodetectors but also a plurality of adjacent photodetectors as input, and can implement various functions other than subtraction through electronic circuits.

[0109] In practice, subtraction and sigmoid function operations can be implemented using analog or digital electronic circuits. While this embodiment illustrates connections between multiple photodetectors and electronic processing units, connections between multiple electronic processing units can also achieve the same effect.

[0110] Each electronic processing unit (630a, 630b) can also store a bias value, amplify the current input from the photodetector, or convert the calculation result into a current value proportional to the result when sending it to the input of the second light source.

[0111] In order to enable such differential mode neural network calculations in the reverse direction, an additional third light source (620a, 620b, 620c, 620d), multiple third A photodetectors (540a, 540c), and multiple third B photodetectors (540b, 540d) are required, as shown in FIG. 5. Light rays (621a, 621b, 622a, 622b, 623a, 623b, 624a, 624b) starting from the third light source (620a, 620b, 620c, 620d) pass through the third lens (534), the second lens (532a, 532b), the spatial light modulator (531a, 532b), and the first lens (530a, 530b) in sequence, and are then gathered on the thirdA light detector (540a, 540c) and the thirdB light detector (540b, 540d).

[0112] The light rays (621a, 621b) originating from the third light source (620a, 620b) are connected to the electronic processing unit (630a) and input. has an intensity proportional to the third light source (620a). The intensity of light coming from the third light source (620b) adjacent to the third light source (620a) is shown in Fig. 5. and are marked as and have the same value has a size proportional to . Similarly, the light from another third light source (620c, 620d) and are marked as and have the same value has a century proportional to it.

[0113] The third light source (620a, 620b) is a pixel of the spatial light modulator (531a). , ) and the lens unit (532a, 534) correspond one-to-one as an object-image relationship. If the spatial light modulator (531a) is spaced apart from the second lens (532a) by the focal length of the second lens and the third lens (534) is spaced apart from the third light source (620a, 620b) by the focal length of the third lens, this object-image relationship is established, and if the distance between the spatial light modulator (531a) and the second lens (532a) changes, the object-image relationship can be satisfied by adjusting the distance between the third lens (534) and the third light source (620a, 620b). Therefore, the light rays (621a, 622a) departing from the third light source (620a, 620b) are focused on the pixels ( , ) and the spatial light modulator (531a), respectively, and face downward and upward. This direction is related to the angle of the light rays departing from the third light source (620a, 620b).

[0114] If the angle of the departing light beam is slightly upward, the downward angle increases when passing through the spatial light modulator (531a). Conversely, if the angle of the departing light beam is downward compared to the original angle, the angle of the light beam changes to an upward angle compared to the original angle when passing through the spatial light modulator (531a). This angle adjustment can be adjusted through the grating period of the prism (451a) or the diffraction grating (450a) used in the third light source (620a, 620b). The angle of the light beam (621a, 622a) when passing through the spatial light modulator (531a) determines the height of the light beam when it reaches the substrate (500) and allows the two light beams (621a, 622a) to reach the third A photodetector (540a) and the third B photodetector (540b), respectively.

[0115] In the same way, light rays (623a, 624a) departing from the third light source (620c, 620d) reach the third A photodetector (540a) and the third B photodetector (540b). Therefore, the signal detected by the third photodetector (540a) is and the signal detected by the 3B photodetector (540b) is proportional to is proportional to. The electronic processing unit (550a) connected to these two photodetectors can perform a subtraction operation with the magnitude of these two signals. A proportional signal size can be obtained.

[0116] As with the forward differential mode calculation above, in the reverse weight representation in Fig. 5, and and Represents the positive part of the reverse weight, and Is and Represents the negative part of the reverse weight. For example, is positive, If is negative, , , , Using You can get results.

[0117]

[0118] Even in the reverse calculation, if there are any number of terms as in Equation 14, the rays corresponding to the (+) term are weighted Set the original model value and the weight After setting to 0, only the rays corresponding to the (+) term can be collected and added to the third A photodetector (540a). Similarly, the rays corresponding to the (-) term can be collected and sent to the third B photodetector (540b) for addition. Therefore, in this way, general neural network operations including negative weights can be performed.

[0119] Each electronic processing unit (550a, 550b) can use the output of not only two surrounding photodetectors but also a plurality of adjacent photodetectors as input, and can implement various functions other than subtraction through electronic circuits.

[0120] Input value in equation 14 in reverse direction Instead, the weighted input stored by the electronic processing unit (550a) can be used can be obtained. The electronic processing unit (550a) uses the values ​​being calculated or stored. Here, silver The pixels corresponding to are spatially separated but set to the same value. In the same way can be obtained.

[0121] Additionally, the spatial light modulator (531a) has a photodetector and an electronic processing unit connected to each pixel to measure the intensity of passing light, store data, exchange information with surrounding pixels, and perform mathematical operations. and In an electronic processing unit close to and Because we can obtain weight sensitivity can be obtained in the same way. , , , , In this way, the optoelectronic module (70) can ultimately perform artificial neural network calculations with any number of input / output nodes or terms in the forward as well as reverse direction, and can process large amounts of parallel data.

[0122] Fig. 6 is a drawing showing an example of cascading the optoelectronic modules of Fig. 5.

[0123] Referring to FIG. 6, the first optical computation layer (901) is repeated and appears in the second optical computation layer (902), and an arbitrary number of neural network layers can be implemented. The output value of the first optical computation layer (901), which is the preceding layer, can be connected to the input of the second optical computation layer (902), which is the succeeding layer.

[0124] Another method of constructing a multilayer neural network in FIG. 5 is to perform multilayer neural network calculations by having light travel back and forth between the first substrate (500) and the second substrate (700) and storing the calculation results in electronic processing units (630a, 630b, 550a, 550b). In this case, each time the light propagates in one direction, the weight values ​​of the spatial light modulators (531a, 532b) must be changed according to the values ​​of each layer. The electronic processing units connected to the weight pixels retrieve the values ​​stored in the memory and update them.

[0125] The present invention provides an optical neural network computer structure and technology that increases the computational speed by performing approximately NXM multiplications and additions at once without complicating the physical structure of the connection lines or generating noise as in an electrical circuit even when the number of input and output terminals increases to N and M, respectively, by implementing an optical computer structure using optical connections instead of purely electronic circuits.

[0126] When the pixel values ​​of the spatial light modulator are already determined, parallel computation is possible because the optical paths from the light source to the photodetector overlap without interfering with each other, and the computation time is very short because the photons arrive at the photodetector at the speed of light.

[0127] The optical neural network computer proposed in the present invention can perform and apply various algorithms, such as neural network learning algorithms such as backpropagation algorithms or auto-correlation algorithms, as well as inference calculations, because light can propagate in both directions.

[0128] Additionally, the time delay in computation occurs in the electronic processing unit, but it is performed in parallel across all outputs, so it can be viewed as equivalent to one instruction cycle of a digital electronic circuit. When configuring L multilayer neural networks, the actual computation speed increases by a factor of N x M x L, since computations are performed simultaneously across all layers.

[0129] Although various embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications can be made by a person having ordinary skill in the art to which the present invention pertains without departing from the gist of the present invention as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present invention.

[0130]

[0131] The present invention provides an optoelectronic module that does not generate noise even when the path of light from a light source to a photodetector overlaps, so that the physical connection does not become complicated like an electrical circuit even when the number of input and output terminals increases, and unlike digital electronic circuits, important parts of the calculation operate in parallel, thereby greatly increasing the calculation speed. In addition, it is a bidirectional structure that can perform calculations while light moves in both the forward and reverse directions, so it is a structure that can implement not only a forward inference algorithm but also a learning function such as a backpropagation algorithm. Therefore, it is expected to be applied to optical computers to enable ultra-small and ultra-high-speed calculations, and it is expected to be usefully applied to pattern recognition, sentence meaning understanding, partial differential equation calculation, and image processing, which were difficult with conventional devices.

[0132] [Explanation of symbols]

[0133] 50, 70: Optoelectronic module

[0134] 400, 500: First substrate

[0135] 430, 700: Second substrate

[0136] 401a, 401b, 501a, 501b: First light source

[0137] 410a, 411a, 412a, 413a, 410b, 411b, 412b, 413b, 415a, 416a, 415b, 416b, 510a, 511a, 512a, 513a, 510b, 511b, 512b, 513b, 621a, 622a, 623a, 624a, 621b, 622b, 623b, 624b: Ray

[0138] 402a, 402b, 530a, 530b: First lens

[0139] 403a, 403b, 531a, 531b: Spatial light modulators

[0140] 404a, 404b, 532a, 532b: Second lens

[0141] 405, 534: Third lens

[0142] 420a, 420b: First photodetector

[0143] 610a, 610c: 1A photodetector

[0144] 610b, 610d: Type 1B photodetector

[0145] 460a, 460b: Second photodetector

[0146] 440a, 440b: Third photodetector

[0147] 540a, 540c: 3A photodetector

[0148] 540b, 540d: 3B photodetector

[0149] 460a, 460b, 710a, 710b: Second photodetector

[0150] 421a, 421b, 441a, 441b, 630a, 630b, 550a, 550b: Electronic Processing Unit

[0151] 422a, 422b, 701a, 701b: Second light source

[0152] 424a, 424b, 620a, 620b, 620c, 620d: Third light source

[0153] 450a: Diffraction grating

[0154] 451a: Prism

[0155] 801, 901: First optical computation layer

[0156] 802, 902: Second optical computation layer

Claims

1. A plurality of first light sources formed at a certain distance from each other; A plurality of first lenses formed to correspond to each of the plurality of first light sources and reducing the divergence angle of light emitted from the corresponding first light sources; A plurality of spatial light modulators each formed corresponding to the plurality of first lenses and controlling the intensity of light passing through the corresponding first lens through respective pixels having preset weights; A plurality of first photodetectors formed spaced apart from the plurality of spatial light modulators and obtaining a current value according to the intensity of light; A lens unit formed between the plurality of spatial light modulators and the plurality of first photodetectors, the lens unit collecting light emitted from different first light sources passing through pixels at the same relative positions of each spatial light modulator onto one first photodetector among the plurality of first photodetectors; A plurality of third light sources formed at a certain distance from the plurality of first light detectors and having light traveling in the opposite direction to the first light source; and A bidirectional optoelectronic module using a spatial light modulator, comprising: a plurality of third photodetectors formed at a predetermined distance from the first light source, collecting light rays irradiated from the plurality of third light sources and passing through the spatial light modulator and the first lens; 2. In paragraph 1, A bidirectional optoelectronic module using a spatial light modulator, wherein the plurality of first photodetectors are formed on the same substrate as the plurality of third light sources, and the plurality of third photodetectors are formed on the same substrate as the plurality of first light sources.

3. In paragraph 1, The above lens part, A plurality of second lenses formed corresponding to each of the plurality of spatial light modulators and concentrating light rays irradiated from the first light source into one point; and A bidirectional optoelectronic module using a spatial light modulator, comprising a third lens for passing all light rays passing through the plurality of second lenses.

4. In paragraph 1, A bidirectional optoelectronic module using a spatial light modulator, wherein the plurality of third light sources are configured to include at least one of a combination of a diffraction grating and a prism or a diffractive optical element.

5. In paragraph 1, It further includes a plurality of electronic processing units that perform a function operation based on the signal of the first photodetector and output the result; A bidirectional optoelectronic module using a spatial light modulator, wherein each electronic processing unit converts and transmits an output signal into an input current of a second light source connected to each electronic processing unit, and the plurality of first photodetectors, the plurality of electronic processing units, and the second light source are formed on a single substrate.

6. In paragraph 5, Each of the above electronic processing units, A bidirectional optoelectronic module using a spatial light modulator, which performs at least one of the following functions: a function of storing a bias value, a function of amplifying a current input from the first photodetector, a function of converting a function operation and the result of the operation into a current value proportional to the result when sending it to the input of the second light source, a memory function, and a data communication function.

7. In paragraph 1, Each spatial light modulator, A bidirectional optoelectronic module using a spatial light modulator, which includes a photodetector and an electronic processing unit connected to each pixel as additional circuitry on the same substrate.

8. In paragraph 1, The above bidirectional optoelectronic module is a bidirectional optoelectronic module using a spatial light modulator that forms a continuous cascade structure.

9. In paragraph 1, The above plurality of first photodetectors, A plurality of first A photodetectors formed spaced apart from the plurality of spatial light modulators and obtaining current values ​​for some of the light passing through the lens unit; and A bidirectional optoelectronic module using a spatial light modulator, comprising: a plurality of first B photodetectors formed on the same substrate as the plurality of first A photodetectors and obtaining current values ​​for light rays remaining except for some of the light rays passing through the lens unit; 10. In paragraph 9, A bidirectional optoelectronic module using a spatial light modulator, further comprising a plurality of electronic processing units connected to the plurality of first A photodetectors and the plurality of first B photodetectors and performing a function operation on the output values ​​of the first A photodetectors and the first B photodetectors and outputting the same.

11. In paragraph 10, A bidirectional optoelectronic module using a spatial light modulator, wherein each electronic processing unit performs a subtraction operation on the output values ​​of the first A photodetector and the first B photodetector and outputs the result.

12. In paragraph 1, The above plurality of third photodetectors are, A plurality of third A photodetectors formed on the same substrate as the plurality of first light sources and spaced apart from the plurality of spatial light modulators, and obtaining current values ​​for some of the light rays irradiated from the third light source and passing through the lens unit; and A bidirectional optoelectronic module using a spatial light modulator, comprising: a plurality of third B photodetectors formed on the same substrate as the plurality of third A photodetectors and obtaining current values ​​for light rays remaining except for some of the light rays passing through the lens unit; 13. In paragraph 12, A bidirectional optoelectronic module using a spatial light modulator, further comprising a plurality of electronic processing units connected to the plurality of 3A photodetectors and the plurality of 3B photodetectors and performing a function operation on the output values ​​of the 3A photodetectors and the 3B photodetectors and outputting the same.

14. In paragraph 13, A bidirectional optoelectronic module using a spatial light modulator, wherein each electronic processing unit performs a subtraction operation on the output values ​​of the third A photodetector and the third B photodetector and outputs the result.

15. An optical neural network computer including a bidirectional optoelectronic module using a spatial light modulator according to paragraph 1.

Images