A Deep Learning Parallel Computation Recognition Device and Method Based on Spatial Light Modulation
By combining spatial light modulators and single-photon detectors, high-efficiency parallel computing for deep learning is achieved, solving the problem of high computational complexity in deep learning algorithms, reducing the computing power requirements of electronic devices, and improving computing speed and processing capabilities.
Patent Information
- Application Number
- CN202311313164.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-11
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-10-11
AI Technical Summary
Deep learning algorithms have high computational complexity in electronic computing, which leads to high demand for computing power in electronic devices and consumes a lot of computing resources.
A deep learning parallel computing recognition device based on spatial light modulation is adopted. It uses a pure amplitude spatial light modulator and an amplitude-phase multiplexed spatial light modulator to perform large-scale matrix-vector multiplication operations, combined with a single-photon detector to perform nonlinear operations, and maps to the spatial and temporal domains.
It significantly improves the speed of parallel computing in deep learning, reduces the computing power requirements of electronic devices, and enhances the processing capabilities of deep learning.
Smart Images

Figure CN117408320B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of information processing technology, and in particular to a deep learning parallel computing recognition device and method based on spatial light modulation. Background Technology
[0002] Deep learning (DL) is a new research direction in the field of machine learning (ML). It was introduced into machine learning to bring it closer to its original goal—artificial intelligence. With the continuous development of deep learning technology, it has achieved many results in image recognition, search technology, data mining, machine translation, natural language processing, multimedia learning, speech recognition, recommendation and personalization technologies, as well as other related fields.
[0003] Deep learning algorithms are based on artificial neural networks, whose powerful capabilities stem from the massive interconnections between a large number of neurons. When deep learning is implemented electronically, large-scale electronic computation is required, and this computation is highly complex, resulting in significant demands on the computing power of electronic devices and consuming substantial computational resources. Summary of the Invention
[0004] To address some or all of the technical problems existing in the prior art, the present invention provides a deep learning parallel computing recognition device and method based on spatial light modulation.
[0005] The technical solution of the present invention is as follows:
[0006] In a first aspect, a deep learning parallel computing recognition device based on spatial light modulation is provided, the device comprising:
[0007] A parallel light source is arranged side by side with a pure amplitude spatial light modulator, an amplitude-phase multiplexed spatial light modulator, and a single photon detector. The parallel light source, the pure amplitude spatial light modulator, the amplitude-phase multiplexed spatial light modulator, and the single photon detector are parallel to each other and coaxial. The parallel light source is used to emit parallel light to the pure amplitude spatial light modulator.
[0008] The pure amplitude spatial light modulator is connected to the processing unit and is used to load a row of matrix elements of the neural network weight coefficient matrix, perform amplitude modulation on the received parallel light according to the loaded matrix, and output the modulated parallel light to the amplitude-phase multiplexed spatial light modulator.
[0009] The amplitude-phase multiplexing spatial light modulator is connected to the processing unit and the distance between it and the single-photon detector is a preset focal length value. It is used to load the lens modulation coefficient vector and the target vector to be processed, to perform amplitude modulation on the received parallel light according to the target vector to be processed, to perform phase modulation on the amplitude-modulated parallel light according to the lens modulation coefficient vector, and to output the modulated parallel light to the single-photon detector.
[0010] The single-photon detector is connected to the processing unit and is used to perform superposition processing on the received parallel light and send the superposition result to the processing unit.
[0011] The processing unit can preload a neural network model, a lens modulation coefficient vector, and a target vector to be processed. It is used to sequentially send the elements of each row of the weight coefficient matrix of each layer of the neural network model to the pure amplitude spatial light modulator, send the lens modulation coefficient vector and the target vector to be processed to the amplitude-phase multiplexed spatial light modulator, perform nonlinear operations on the superposition results received each time, and update the target vector to be processed and obtain the deep learning parallel computing recognition results based on the nonlinear operation results.
[0012] In some possible implementations, the parallel light source is an incoherent light source.
[0013] In some possible implementations, an output unit is also included, connected to the processing unit, for receiving and outputting the deep learning parallel computing recognition results sent by the processing unit.
[0014] In some possible implementations, the processing unit is a microcontroller unit.
[0015] Secondly, a deep learning parallel computing recognition method based on spatial light modulation is also provided. This method utilizes the aforementioned deep learning parallel computing recognition device based on spatial light modulation and includes the following steps:
[0016] Step S1: Obtain the neural network model and the target vector to be processed, and load the neural network model and the target vector to be processed into the processing unit;
[0017] Step S2: Obtain the lens modulation coefficient vector, and use the processing unit to load the lens modulation coefficient vector onto the phase modulation of the amplitude-phase multiplexed spatial light modulator;
[0018] Step S3: Set the layer number variable k and the output vector Y, and set k = 1;
[0019] Step S4: Determine if k ≤ K is true. If so, then... Proceed to step S5; otherwise, proceed to step S13. K represents the number of layers in the neural network model. Represents the empty set;
[0020] Step S5: Determine the neural network weight coefficient matrix of the k-th layer of the neural network model and its number of rows and columns. Transform the target vector to be processed into a column vector with the same number of elements as the number of rows of the neural network weight coefficient matrix. Load the transformed column vector onto the amplitude modulation of the amplitude phase multiplexing spatial light modulator. Set the row number variable n and make n = 1.
[0021] Step S6: Determine whether n≤N is true. If yes, proceed to step S7; otherwise, proceed to step S12. N represents the row number of the weight coefficient matrix of the current k-th layer of the neural network.
[0022] Step S7: Load the nth row of the neural network weight coefficient matrix of the kth layer onto the amplitude modulation of the pure amplitude spatial light modulator;
[0023] Step S8: Parallel light is emitted to the pure amplitude spatial light modulator using a parallel light source, so that the parallel light is focused onto the single photon detector after being modulated by the amplitude of the pure amplitude spatial light modulator, the amplitude of the amplitude-phase multiplexed spatial light modulator, and the phase of the amplitude-phase multiplexed spatial light modulator.
[0024] Step S9: Use a single-photon detector to perform single-pixel photoelectric detection on the received parallel light, obtain the superposition result of the parallel light, and output it to the processing unit.
[0025] Step S10: Perform nonlinear operations on the superposition results using the processing unit to obtain a calculation result;
[0026] Step S11: Update the output vector as a new element of the output vector Y using the obtained operation result, so that n = n + 1, and return to step S6;
[0027] Step S12: Assign the output vector Y to the target vector X to be processed, such that k = k + 1, and return to step S4;
[0028] Step S13: Output vector Y.
[0029] In some possible implementations, the m-th parallel light output by the pure amplitude spatial light modulator Represented as:
[0030]
[0031] Among them, w nm The nth row and mth column of the neural network weight coefficient matrix represents the element, e represents the natural constant, i is the imaginary unit, w represents the angular frequency of the parallel light, and t represents the time variable.
[0032] In some possible implementations, the m-th parallel light output by the amplitude-phase multiplexed spatial light modulator Represented as:
[0033]
[0034] Among them, w nm This represents the element in the nth row and mth column of the neural network weight coefficient matrix, x. m Let represent the m-th element of the target vector to be processed, e represent the natural constant, i represent the imaginary unit, w represent the angular frequency of the parallel light, and t represent the time variable. This represents the m-th element of the lens modulation coefficient vector.
[0035] In some possible implementations, the superposition of parallel beams output by a single-photon detector is represented as:
[0036]
[0037] Among them, s n This represents the result of the superposition of parallel light, where M represents the number of columns in the neural network weight coefficient matrix, and w nm This represents the element in the nth row and mth column of the neural network weight coefficient matrix, x. m This represents the m-th element of the target vector to be processed.
[0038] In some possible implementations, the superposition result is represented by a nonlinear operation as follows:
[0039] y n =activation(s n +b n )
[0040] Among them, y n The superposition result s n The corresponding calculation result, b n This indicates the corresponding superposition result s n The bias coefficient is denoted by , and activation(·) represents the activation function.
[0041] In some possible implementations, the activation function is one of the following: Sigmoid, Tanh, ReLU, LReLU, PReLU, and Swish activation functions.
[0042] The main advantages of the technical solution of this invention are as follows:
[0043] The deep learning parallel computing recognition method based on spatial light modulation of the present invention maps large-scale matrix-vector multiplication operations to the spatial and temporal domains by using a combination of a pure amplitude spatial light modulator and an amplitude-phase multiplexed spatial light modulator, thereby achieving large-scale matrix-vector multiplication parallel operations. This significantly improves the parallel computing speed of deep learning. Furthermore, the rapid switching of the spatial light modulator enables multi-output and multi-layer neural network operations. The single-photon detector allows for point-by-point nonlinear operations on the results of matrix-vector multiplication, which greatly reduces the computational complexity of deep learning, reduces the computational power requirements of electronic devices, and improves the processing power of deep learning. Attached Figure Description
[0044] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and constitute a part of this invention, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:
[0045] Figure 1 This is a schematic diagram of the structure of a deep learning parallel computing recognition device based on spatial light modulation according to an embodiment of the present invention.
[0046] Figure 2 This is a schematic diagram of another deep learning parallel computing recognition device based on spatial light modulation according to an embodiment of the present invention.
[0047] Figure 3 This is a flowchart of a deep learning parallel computing recognition method based on spatial light modulation according to an embodiment of the present invention.
[0048] Explanation of reference numerals in the attached figures:
[0049] 1-Parallel light source, 2-Pure amplitude spatial light modulator, 3-Amplitude and phase multiplexing spatial light modulator, 4-Single photon detector, 5-Processing unit, 6-Output unit. Detailed Implementation
[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0051] The technical solutions provided by the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
[0052] refer to Figure 1In a first aspect, an embodiment of the present invention provides a deep learning parallel computing recognition device based on spatial light modulation, the device comprising:
[0053] Parallel light source 1, pure amplitude spatial light modulator 2, amplitude phase multiplexing spatial light modulator 3 and single photon detector 4 are arranged side by side in sequence. Parallel light source 1, pure amplitude spatial light modulator 2, amplitude phase multiplexing spatial light modulator 3 and single photon detector 4 are parallel to each other and coaxial. Parallel light source 1 is used to emit parallel light to pure amplitude spatial light modulator 2.
[0054] The pure amplitude spatial light modulator 2 is connected to the processing unit 5. It is used to load a row of matrix elements of the neural network weight coefficient matrix, perform amplitude modulation on the received parallel light according to the loaded matrix, and output the modulated parallel light to the amplitude-phase multiplexed spatial light modulator 3.
[0055] Amplitude-phase multiplexing spatial light modulator 3 is connected to processing unit 5 and the distance between it and single-photon detector 4 is a preset focal length value. It is used to load lens modulation coefficient vector and target vector to be processed, perform amplitude modulation on the received parallel light according to the target vector to be processed, perform phase modulation on the amplitude-modulated parallel light according to the lens modulation coefficient vector, and output the modulated parallel light to single-photon detector 4.
[0056] The single-photon detector 4 is connected to the processing unit 5 and is used to perform superposition processing on the received parallel light and send the superposition result to the processing unit 5.
[0057] The processing unit 5 is capable of preloading a neural network model, a lens modulation coefficient vector, and a target vector to be processed. It is used to send the elements of each row of the weight coefficient matrix of each layer of the neural network model to the pure amplitude spatial light modulator 2 in sequence, and to send the lens modulation coefficient vector and the target vector to be processed to the amplitude-phase multiplexing spatial light modulator 3. It performs nonlinear operations on the superposition results received each time, and updates the target vector to be processed and obtains the deep learning parallel computing recognition results based on the nonlinear operation results.
[0058] An embodiment of the present invention provides a deep learning parallel computing recognition device based on spatial light modulation, which is used to realize the parallel computing recognition of target vectors by a deep learning neural network model.
[0059] Specifically, in use, this deep learning parallel computing recognition device based on spatial light modulation first loads the deep learning neural network model, lens modulation coefficient vector, and target vector to be processed into the processing unit. Then, starting from the first layer of the neural network model, it determines the target vector to be processed corresponding to that layer. The target vector to be processed corresponding to the first layer is the target vector to be processed loaded into the processing unit. The processing unit converts the target vector to be processed into a matrix and loads it onto the amplitude modulation of the amplitude-phase multiplexed spatial light modulator, and converts the lens modulation coefficient vector into a matrix and loads it onto the phase modulation of the amplitude-phase multiplexed spatial light modulator to achieve the digital recognition of a specific focal length. The image is processed using a lens function. Then, starting from the first row vector of the neural network weight coefficient matrix corresponding to that layer, the row vectors of the neural network weight coefficient matrix corresponding to that layer are sequentially converted into matrices and loaded onto the amplitude modulation of the pure amplitude spatial light modulator. After each conversion of the neural network weight coefficient matrix row vectors into matrices and loading them onto the pure amplitude spatial light modulator, parallel light is emitted from a parallel light source. This parallel light undergoes amplitude modulation by the pure amplitude spatial light modulator, amplitude modulation by the amplitude-phase multiplexed spatial light modulator, and phase modulation by the phase-phase multiplexed spatial light modulator, before converging onto a single-photon detector. The single-photon detector then performs single-pixel photoelectric detection to obtain the image. The superposition results of all parallel light rays are used to perform multiplication between the row vectors of the current neural network weight coefficient matrix and the target vector to be processed. The superposition results are sent to the processing unit using a single-photon detector. The processing unit performs nonlinear operations on the superposition results to obtain a calculation result corresponding to the row vectors of the current neural network weight coefficient matrix. By sequentially converting the row vectors of the neural network weight coefficient matrix corresponding to the current layer into matrices and loading them onto the amplitude modulation of a pure amplitude spatial light modulator, and performing the above processing after each loading, all calculation results corresponding to all row vectors of the neural network weight coefficient matrix can be obtained, thus completing the single-layer calculation of the current layer of the neural network model. The calculation results corresponding to all row vectors of the neural network weight coefficient matrix are the single-layer calculation results. Then, the calculation results of the current layer are used as the target vector to be processed for the next layer of the neural network model, and the neural network weight coefficient matrix corresponding to the next layer of the neural network model is determined. Based on the determined target vector to be processed and the neural network weight coefficient matrix, the single-layer calculation process described above is used to complete the single-layer calculation of the next layer of the neural network model. The calculations of each layer of the neural network model are completed sequentially according to the above processing method. After multiple calculations, the calculation result output by the last layer is the parallel computing recognition result of the deep learning neural network.
[0060] An embodiment of the present invention provides a deep learning parallel computing recognition device based on spatial light modulation. By combining a pure amplitude spatial light modulator and an amplitude-phase multiplexed spatial light modulator, large-scale matrix-vector multiplication operations are mapped to the spatial and temporal domains to achieve large-scale matrix-vector multiplication parallel operations. This can significantly improve the speed of deep learning parallel computing. Furthermore, the rapid switching of the spatial light modulator enables multi-output and multi-layer neural network operations. Through a single-photon detector, nonlinear operations can be performed point-by-point on the results of matrix-vector multiplication, which can greatly reduce the computational complexity of deep learning, reduce the computing power requirements of electronic devices, and improve the deep learning processing capability.
[0061] In one embodiment of the present invention, the preset focal length value is specifically set according to the actual situation, wherein the focal length corresponding to the loaded lens modulation coefficient vector is the preset focal length value.
[0062] Furthermore, in one embodiment of the present invention, the parallel light source 1 is an incoherent light source to avoid interference between different beams of parallel light that could affect the processing results.
[0063] refer to Figure 2 Furthermore, in one embodiment of the present invention, the deep learning parallel computing recognition device further includes an output unit 6 connected to the processing unit 5, for receiving and outputting the deep learning parallel computing recognition result sent by the processing unit 5.
[0064] Thus, when an external device needs to obtain the recognition results of deep learning parallel computing, the external device can be directly connected to the output unit to obtain the recognition results of deep learning parallel computing.
[0065] refer to Figure 2 In one embodiment of the present invention, the processing unit may also be connected to a parallel light source, and the processing unit is used to control the parallel light source to emit parallel light.
[0066] By connecting the processing unit to the parallel light source, the parallel light source can be turned on and off by the processing unit to control the parallel light source to emit parallel light at a specific time. This ensures that after the row vector of the neural network weight coefficient matrix is loaded into the pure amplitude spatial light modulator, the parallel light source can emit parallel light to the pure amplitude spatial light modulator in a timely manner.
[0067] Furthermore, in one embodiment of the present invention, the processing unit is a microcontroller unit (MCU).
[0068] By using a microcontroller unit as the processing unit, the above functional requirements can be met, and the deep learning parallel computing recognition device can be made lightweight and low-power.
[0069] refer to Figure 3Secondly, an embodiment of the present invention also provides a deep learning parallel computing recognition method based on spatial light modulation. This method is implemented using the aforementioned deep learning parallel computing recognition device and includes the following steps S1-S13:
[0070] Step S1: Obtain the neural network model and the target vector to be processed, and load the neural network model and the target vector to be processed into the processing unit.
[0071] In one embodiment of the present invention, a neural network model and a target vector to be processed are obtained according to the actual deep learning process, and the neural network model and the target vector to be processed are loaded into the processing unit.
[0072] Step S2: Obtain the lens modulation coefficient vector and use the processing unit to load the lens modulation coefficient vector onto the phase modulation of the amplitude-phase multiplexed spatial light modulator.
[0073] In one embodiment of the present invention, based on the installation distance between the amplitude-phase multiplexed spatial light modulator and the single-photon detector, a lens modulation coefficient vector corresponding to the aforementioned preset focal length value is obtained. The lens modulation coefficient vector is loaded into the processing unit, and the processing unit sends the lens modulation coefficient vector to the amplitude-phase multiplexed spatial light modulator. Then, the lens modulation coefficient vector is loaded onto the phase modulation of the amplitude-phase multiplexed spatial light modulator.
[0074] Step S3: Set the layer number variable k and the output vector Y, and set k = 1.
[0075] In one embodiment of the present invention, a layer number variable k and an output vector Y are set in the processing unit, and the layer number variable k is initialized so that k = 1.
[0076] Step S4: Determine if k ≤ K is true. If so, then... Proceed to step S5; otherwise, proceed to step S13. K represents the number of layers in the neural network model. This represents the empty set.
[0077] In one embodiment of the present invention, this judgment step can be implemented by a processing unit. When k≤K, the output vector is initialized to... Then proceed to step S5. When k > K, proceed to step S13 to output the parallel computing recognition result of the deep learning neural network.
[0078] Step S5: Determine the neural network weight coefficient matrix of the k-th layer of the neural network model and its number of rows and columns. Transform the target vector to be processed into a column vector with the same number of elements as the number of rows of the neural network weight coefficient matrix. Load the transformed column vector onto the amplitude modulation of the amplitude phase multiplexing spatial light modulator. Set the row number variable n and make n = 1.
[0079] In one embodiment of the present invention, the processing unit converts the target vector to be processed into a column vector with the same number of elements as the number of rows of the neural network weight coefficient matrix, and sends the converted column vector to the amplitude phase multiplexing spatial light modulator, and then loads the column vector onto the amplitude modulation of the amplitude phase multiplexing spatial light modulator.
[0080] Specifically, the weight coefficient matrix of the k-th layer of the neural network model is defined as follows:
[0081]
[0082] The target vector to be processed can then be represented as a column vector as follows:
[0083] X = [x1, x2, ..., x M ] T
[0084] Among them, W1, W2, ..., W N Let W represent the row vector of the neural network weight coefficient matrix, N represent the number of rows in the neural network weight coefficient matrix, and M represent the number of columns in the neural network weight coefficient matrix. NM This represents the element in the Nth row and Mth column of the neural network weight coefficient matrix W, x1, x2, ..., x M This represents the first to the Mth elements of the target vector X to be processed.
[0085] Step S6: Determine whether n≤N is true. If yes, proceed to step S7; otherwise, proceed to step S12. N represents the row number of the neural network weight coefficient matrix of the current k-th layer.
[0086] In one embodiment of the present invention, the determination step can be implemented by a processing unit.
[0087] Step S7: Load the nth row of the neural network weight coefficient matrix of the kth layer onto the amplitude modulation of the pure amplitude spatial light modulator.
[0088] In one embodiment of the present invention, the processing unit sends the nth row of the weight coefficient matrix of the k-th layer neural network to the pure amplitude spatial light modulator, and then loads the nth row of the weight coefficient matrix of the k-th layer neural network onto the amplitude modulation of the pure amplitude spatial light modulator.
[0089] Specifically, based on the neural network weight coefficient matrix W set in the above steps, W is... n =[w n1 ,w n2 ,…,w nM The amplitude modulation is applied to the pure amplitude spatial light modulator.
[0090] Step S8: Parallel light is emitted to the pure amplitude spatial light modulator using a parallel light source, so that the parallel light is focused onto the single photon detector after being modulated by the amplitude of the pure amplitude spatial light modulator, the amplitude of the amplitude-phase multiplexed spatial light modulator, and the phase of the amplitude-phase multiplexed spatial light modulator.
[0091] refer to Figure 1 In one embodiment of the present invention, parallel light is emitted from a parallel light source to a pure amplitude spatial light modulator. Since the parallel light source, the pure amplitude spatial light modulator, the amplitude-phase multiplexed spatial light modulator, and the single-photon detector are arranged sequentially, the parallel light first enters the pure amplitude spatial light modulator, undergoes amplitude modulation by the pure amplitude spatial light modulator, and is then output. The output parallel light then enters the amplitude-phase multiplexed spatial light modulator, undergoes amplitude modulation and phase modulation by the amplitude-phase multiplexed spatial light modulator, and is then output. Because the distance between the amplitude-phase multiplexed spatial light modulator and the single-photon detector is a preset focal length value, after passing through the lens modulation coefficient vector, the parallel light output from the amplitude-phase multiplexed spatial light modulator converges onto the single-photon detector.
[0092] Define the modulation matrix of a pure amplitude spatial light modulator and an amplitude-phase multiplexed spatial light modulator as having the same size, with I rows and J columns. Let I·J=M, and represent the modulation matrix of the spatial light modulator as a vector.
[0093] Based on the above steps, the parallel light emitted by the parallel light source is amplitude modulated by a pure amplitude spatial light modulator, resulting in the output of M parallel lights. The m-th parallel light... It can be represented as:
[0094]
[0095] Furthermore, the lens modulation coefficient vector is set as... The parallel light output from the pure amplitude spatial light modulator is modulated by both amplitude and phase modulation by the amplitude-phase multiplexed spatial light modulator, resulting in M parallel lights. The m-th parallel light... It can be represented as:
[0096]
[0097] Among them, w nm x represents the element in the nth row and mth column of the weight coefficient matrix W of the neural network. m Let represent the m-th element of the target vector X to be processed, e represent the natural constant, i represent the imaginary unit, w represent the angular frequency of the parallel light, and t represent the time variable. This represents the m-th element of the lens modulation coefficient vector Φ.
[0098] Step S9: Use a single-photon detector to perform single-pixel photoelectric detection on the received parallel light, obtain the superposition result of the parallel light, and output it to the processing unit.
[0099] In one embodiment of the present invention, a single-photon detector is used to perform single-pixel photoelectric detection on multiple parallel lights, and the superposition result of multiple parallel lights can be obtained after photoelectric detection.
[0100] Specifically, based on the above steps, the superposition result of parallel light can be expressed as:
[0101]
[0102] Among them, s n This represents the superposition result of M parallel lights output by the amplitude-phase multiplexed spatial light modulator.
[0103] Step S10: Use the processing unit to perform nonlinear operations on the superposition results to obtain a calculation result.
[0104] In one embodiment of the present invention, a nonlinear operation is performed on the received superposition result in the processing unit to obtain an operation result, wherein the nonlinear operation method is specifically set according to the actual situation.
[0105] Specifically, the superposition result is represented by a nonlinear operation as follows:
[0106] y n =activation(s n +b n )
[0107] Among them, y n The superposition result s n The corresponding calculation result, b n This indicates the corresponding superposition result s n The bias coefficient is denoted by , and activation(·) represents the activation function. The activation function is set according to the actual needs, such as one of the activation functions Sigmoid, Tanh, ReLU, LReLU, PReLU and Swish.
[0108] Step S11: Update the output vector as a new element of the output vector Y using the obtained operation result, so that n = n + 1, and return to step S6.
[0109] In one embodiment of the present invention, the updated output vector Y can be expressed as:
[0110] Y = [Y; y n ]
[0111] Step S12: Assign the output vector Y to the target vector X to be processed, such that k = k + 1, and return to step S4.
[0112] In one embodiment of the present invention, since the output of the previous layer of the neural network model is the input of the next layer, it is necessary to update the target vector to be processed in the next layer according to the calculation result of the previous layer.
[0113] Specifically, the updated target vector to be processed can be represented as:
[0114] X = Y = [y1y2…y] N ]
[0115] Step S13: Output vector Y.
[0116] In one embodiment of the present invention, the output vector Y output in step S13 is the parallel computing recognition result of the deep learning neural network.
[0117] An embodiment of the present invention provides a deep learning parallel computing recognition method based on spatial light modulation. By combining a pure amplitude spatial light modulator and an amplitude-phase multiplexed spatial light modulator, large-scale matrix-vector multiplication operations are mapped to the spatial and temporal domains to achieve large-scale matrix-vector multiplication parallel operations. This significantly improves the speed of deep learning parallel computing. Furthermore, the rapid switching of the spatial light modulator enables multi-output and multi-layer neural network operations. Through a single-photon detector, nonlinear operations can be performed point-by-point on the results of matrix-vector multiplication, which greatly reduces the computational complexity of deep learning, reduces the computing power requirements of electronic devices, and improves the deep learning processing capability.
[0118] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Additionally, the terms "front," "back," "left," "right," "upper," and "lower" in this document refer to the placement shown in the accompanying drawings.
[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A deep learning parallel computing recognition device based on spatial light modulation, characterized in that, include: A parallel light source is arranged side by side with a pure amplitude spatial light modulator, an amplitude-phase multiplexed spatial light modulator, and a single photon detector. The parallel light source, the pure amplitude spatial light modulator, the amplitude-phase multiplexed spatial light modulator, and the single photon detector are parallel to each other and coaxial. The parallel light source is used to emit parallel light to the pure amplitude spatial light modulator. The pure amplitude spatial light modulator is connected to the processing unit and is used to load a row of matrix elements of the neural network weight coefficient matrix, perform amplitude modulation on the received parallel light according to the loaded matrix, and output the modulated parallel light to the amplitude-phase multiplexed spatial light modulator. The amplitude-phase multiplexing spatial light modulator is connected to the processing unit and the distance between it and the single-photon detector is a preset focal length value. It is used to load the lens modulation coefficient vector and the target vector to be processed, to perform amplitude modulation on the received parallel light according to the target vector to be processed, to perform phase modulation on the amplitude-modulated parallel light according to the lens modulation coefficient vector, and to output the modulated parallel light to the single-photon detector. The single-photon detector is connected to the processing unit and is used to perform superposition processing on the received parallel light and send the superposition result to the processing unit. The processing unit can preload a neural network model, a lens modulation coefficient vector, and a target vector to be processed. It is used to sequentially send the elements of each row of the weight coefficient matrix of each layer of the neural network model to the pure amplitude spatial light modulator, send the lens modulation coefficient vector and the target vector to be processed to the amplitude-phase multiplexed spatial light modulator, perform nonlinear operations on the superposition results received each time, and update the target vector to be processed and obtain the deep learning parallel computing recognition results based on the nonlinear operation results.
2. The deep learning parallel computing recognition device based on spatial light modulation according to claim 1, characterized in that, The parallel light source is an incoherent light source.
3. The deep learning parallel computing recognition device based on spatial light modulation according to claim 1, characterized in that, It also includes an output unit connected to the processing unit, used to receive and output the deep learning parallel computing recognition results sent by the processing unit.
4. The deep learning parallel computing recognition device based on spatial light modulation according to claim 1, characterized in that, The processing unit is a microcontroller unit.
5. A deep learning parallel computing recognition method based on spatial light modulation, characterized in that, The method is implemented using the deep learning parallel computing recognition device as described in any one of claims 1-4, and includes the following steps: Step S1: Obtain the neural network model and the target vector to be processed, and load the neural network model and the target vector to be processed into the processing unit; Step S2: Obtain the lens modulation coefficient vector, and use the processing unit to load the lens modulation coefficient vector onto the phase modulation of the amplitude-phase multiplexed spatial light modulator; Step S3: Set the layer number variable k and the output vector Y, and set k = 1; Step S4: Determine if k ≤ K is true. If so, then... Proceed to step S5; otherwise, proceed to step S13. K represents the number of layers in the neural network model. Represents the empty set; Step S5: Determine the neural network weight coefficient matrix of the k-th layer of the neural network model and its number of rows and columns. Transform the target vector to be processed into a column vector with the same number of elements as the number of rows of the neural network weight coefficient matrix. Load the transformed column vector onto the amplitude modulation of the amplitude phase multiplexing spatial light modulator. Set the row number variable n and make n = 1. Step S6: Determine whether n≤N is true. If yes, proceed to step S7; otherwise, proceed to step S12. N represents the row number of the weight coefficient matrix of the current k-th layer of the neural network. Step S7: Load the nth row of the neural network weight coefficient matrix of the kth layer onto the amplitude modulation of the pure amplitude spatial light modulator; Step S8: Parallel light is emitted to the pure amplitude spatial light modulator using a parallel light source, so that the parallel light is focused onto the single photon detector after being modulated by the amplitude of the pure amplitude spatial light modulator, the amplitude of the amplitude-phase multiplexed spatial light modulator, and the phase of the amplitude-phase multiplexed spatial light modulator. Step S9: Use a single-photon detector to perform single-pixel photoelectric detection on the received parallel light, obtain the superposition result of the parallel light, and output it to the processing unit. Step S10: Perform nonlinear operations on the superposition results using the processing unit to obtain a calculation result; Step S11: Update the output vector as a new element of the output vector Y using the obtained operation result, so that n = n + 1, and return to step S6; Step S12: Assign the output vector Y to the target vector X to be processed, such that k = k + 1, and return to step S4; Step S13: Output vector Y.
6. The deep learning parallel computing recognition method based on spatial light modulation according to claim 5, characterized in that, The m-th parallel light output by the pure amplitude spatial light modulator Represented as: Among them, w nm The nth row and mth column of the neural network weight coefficient matrix represents the element, e represents the natural constant, i is the imaginary unit, w represents the angular frequency of the parallel light, and t represents the time variable.
7. The deep learning parallel computing recognition method based on spatial light modulation according to claim 5, characterized in that, The m-th parallel light output from the amplitude-phase multiplexed spatial light modulator Represented as: Among them, w nm This represents the element in the nth row and mth column of the neural network weight coefficient matrix, x. m Let represent the m-th element of the target vector to be processed, e represent the natural constant, i represent the imaginary unit, w represent the angular frequency of the parallel light, and t represent the time variable. This represents the m-th element of the lens modulation coefficient vector.
8. The deep learning parallel computing recognition method based on spatial light modulation according to claim 5, characterized in that, The superposition of parallel beams output by a single-photon detector is expressed as: Among them, s n This represents the result of the superposition of parallel light, where M represents the number of columns in the neural network weight coefficient matrix, and w nm This represents the element in the nth row and mth column of the neural network weight coefficient matrix, x. m This represents the m-th element of the target vector to be processed.
9. The deep learning parallel computing recognition method based on spatial light modulation according to claim 8, characterized in that, The superposition result is represented by a nonlinear operation as follows: y n =activation(s n +b n ) Among them, y n The superposition result s n The corresponding calculation result, b n This indicates the corresponding superposition result s n The bias coefficient is denoted by , and activation(·) represents the activation function.
10. The deep learning parallel computing recognition method based on spatial light modulation according to claim 9, characterized in that, The activation function is one of the following: Sigmoid, Tanh, ReLU, LReLU, PReLU, and Swish activation functions.
Citation Information
Patent Citations
Photonic neural network
CN112232504A
Optical modulation neuron and all-optical diffraction neural network method for signal processing
CN115545173A
