Multilayer optoelectronic convolutional neural network system and method based on optical parallel method

Abstract

The present invention provides a multi-layer optoelectronic convolutional neural network system and method based on an optical parallel method, which relates to the field of optoelectronic neural network computing and includes: a multi-wavelength laser generation and modulation module, which is used to generate N groups of multi-wavelength laser signals representing N convolutional layers, perform information modulation and transmission path management, so that the wavelengths of each group of multi-wavelength laser signals carry the data to be convolved in different convolutional regions of the same layer; a parallel optical convolution module, which is used to perform parallel convolution on N groups of multi-wavelength signals to be convolved in the optical operation domain, and realize the forward propagation of the multi-layer network by increasing the parallel data flow; an optoelectronic conversion module, which is used to convert the light intensity result of each wavelength obtained by convolution into a voltage signal; an electrical processing and control module, which is used to post-process the voltage signal, and update the data to be convolved according to the numerical result of the post-processing and the required data obtained; a data storage module, which is used to store the operation results of each layer in the forward propagation and transmit the data to be convolved in the current parallel convolution operation.

Description

Technical Field

[0001] The present invention relates to the technical field of optoelectronic neural network computing, and particularly relates to a multi-layer optoelectronic convolutional neural network system and method based on an optical parallel method. Background Art

[0002] As one of the representative algorithms in current deep learning technologies, the convolutional neural network is widely used in tasks such as object detection and image and video recognition. Feature extraction and local connection are achieved through convolutional operations, non-linear activation increases the non-linear representation ability of the network, and pooling reduces the risk of overfitting by compressing parameters. At present, with the continuous development of deep learning technologies and the increasing complexity of data tasks, the parameter scale and number of layers of the convolutional neural network are constantly expanding, bringing huge challenges to the training and inference efficiency of the convolutional neural network. Summary of the Invention

[0003] In view of the above problems, the present invention provides a multi-layer optoelectronic convolutional neural network system and method based on an optical parallel method, which are used to at least partially solve the above technical problems.

[0004] According to a first aspect of an embodiment of the present invention, there is provided a multi-layer optoelectronic convolutional neural network system based on an optical parallel method, including: a multi-wavelength laser generation and modulation module configured to generate N groups of multi-wavelength laser signals representing N convolutional layers, perform information modulation and transmission path management on the N groups of multi-wavelength laser signals, so that the wavelengths in each group of multi-wavelength laser signals carry the data to be convolved in different convolutional regions of the same layer, and obtain N groups of multi-wavelength signals to be convolved, where N is a positive integer greater than 1; a parallel optical convolution module configured to perform parallel convolution operations on the received N groups of multi-wavelength signals to be convolved in the optical operation domain, and realize the forward propagation of the multi-layer neural network by increasing the parallel data flow rate, and obtain the light intensity result of each wavelength; a photoelectric conversion module configured to output the received light intensity result of each wavelength in the form of a voltage signal; an electrical processing and control module configured to perform post-processing on the received voltage signal to obtain the numerical result of the convolution operation, and control the multi-wavelength laser generation and modulation module according to the numerical result and the required data obtained from the outside to update the input data to be convolved; and a data storage module configured to store the operation results of each layer of the forward propagation and transmit the data to be convolved required for the current parallel convolution operation.

[0005] According to an embodiment of the present invention, the multi-wavelength laser generation and modulation module is sequentially connected to a multi-wavelength laser array, a modulator array, and a wavelength division multiplexer array; the multi-wavelength laser array includes N laser units for generating N groups of multi-wavelength laser signals representing N convolutional layers, where each laser unit contains L1, L2,..., L NA plurality of multi-wavelength lasers for generating multi-wavelength laser signals carrying data of different regions in the Nth layer of convolutional operations; a modulator array including N modulator units for respectively modulating N groups of laser signals of different wavelengths and loading N groups of information onto the N groups of multi-wavelength laser signals, such that each wavelength of the laser signal carries corresponding data to be convolved. Among them, each modulator unit contains the same corresponding number of L1, L2, …, L N modulators; a wavelength division multiplexer array including N wavelength division multiplexers for wavelength division multiplexing the data to be convolved in different operation regions of the same convolutional layer but represented by different wavelengths into the same transmission path to obtain N groups of multi-wavelength data to be convolved signals; where N is a positive integer greater than 1, and L1, L2, …, L N are all positive integers greater than or equal to 1.

[0006] According to an embodiment of the present invention, the parallel optical convolution module includes an input optical circulator, a parallel optical convolution unit, and an output optical circulator connected in sequence; the input optical circulator includes a first optical circulator for separating the forward light input and the backward light output on the channel to prevent the backward light output from entering the optical path of the forward light input; the parallel optical convolution unit is used to implement parallel operations on the data to be convolved in different regions of each convolutional layer in the N convolutional layers; the output optical circulator includes a second optical circulator for separating the backward light input and the forward light output on the channel to prevent the forward light output from entering the optical path of the backward light input; a beam splitting device is further provided between the output of the multi-wavelength laser generation modulation module and the forward light input and the backward light input of the parallel optical convolution module, and the beam splitting device is used to evenly divide the output of the multi-wavelength laser generation modulation module and input it into the forward light input port and the backward light input port of the parallel optical convolution module.

[0007] According to an embodiment of the present invention, the parallel optical convolution module includes a parallel optical convolution unit; the parallel optical convolution unit is used to implement parallel operations on the data to be convolved in different regions of each convolutional layer in the N convolutional layers.

[0008] According to an embodiment of the present invention, the input of the output of the multi-wavelength laser generation modulation module to the parallel optical convolution module is the same or different for each convolutional layer in the N convolutional layers, including single forward light input, single backward light input, or simultaneous forward and backward input.

[0009] According to an embodiment of the present invention, the parallel optical convolution unit includes: one of an optical convolution arithmetic unit based on a Mach-Zehnder interferometer, an optical convolution arithmetic unit based on a microring filter, an optical convolution arithmetic unit based on a multimode interference coupler, an optical convolution arithmetic unit based on diffractive optics, an optical convolution arithmetic unit based on interference optics, an optical convolution arithmetic unit based on wavelength division multiplexing, and an optical convolution arithmetic unit based on a tunable optical amplifier and an optical attenuator.

[0010] According to an embodiment of the present invention, the optoelectronic conversion module includes a wavelength division multiplexer array, a photodetector array, and a transimpedance amplifier array connected in sequence; the wavelength division multiplexer array includes N wavelength division multiplexers for separating the optical intensity results obtained by parallel operation from one channel into each individual channel; the photodetector array includes N groups of detector units, where each group of detector units respectively includes L1, L2, …, L N photodetectors in number, for performing optical intensity detection on the optical intensity results of each wavelength after convolution operation based on optoelectronic conversion to convert the optical intensity result number into a photocurrent signal; the transimpedance amplifier array includes N groups of transimpedance amplifier units, where each transimpedance amplifier unit respectively includes L1, L2, …, L N transimpedance amplifiers in number, for amplifying the photocurrent signal output by the corresponding detector unit and converting it into a voltage signal.

[0011] According to an embodiment of the present invention, the electrical processing control module includes a first register and an electrical post-processing unit, a first register, a discriminator, and a modulator control unit connected in sequence, and the input of the first register is connected to the output of the discriminator; the electrical post-processing unit is used for post-processing the input voltage signal, and obtaining the numerical value of the convolution operation through electrical post-processing transformation operation; the first register is used for transferring the result of the post-processed convolution operation data that is not involved in the next convolution operation to the data storage module, and retrieving the data to be convolved required for the next operation from the data storage module; the discriminator is used to implement the non-linear activation and max pooling operations in the convolutional neural network to obtain the data after non-linear activation and pooling processing and the position information corresponding to the data change; the second register is used for recording the position information where the non-linear activation and pooling operation data changes and transferring it to the data storage module; the modulator control unit is used for generating a corresponding regulation voltage according to the result obtained after a complete operation, and using this result as the initial input value for the next operation.

[0012] According to an embodiment of the present invention, the data storage module includes: an initial input storage unit for storing the data to be convolved of the initial input and sending corresponding data to an external module according to calculation requirements; an intermediate layer data storage unit for storing the data generated during each complete operation in the middle and the position information where the corresponding data changes, for subsequent extraction of the generated data and position information for numerical derivative operation for backpropagation, and sending corresponding data to an external module according to calculation requirements; a terminal data storage unit for storing the final data and the corresponding position information after the last layer completes convolution, non-linear activation, and pooling operations.

[0013] The second aspect of the embodiments of the present invention provides a parallel operation method, which is implemented based on the above system and includes: using a parallel optical convolution module to perform parallel convolution operations on N groups of multi-wavelength signals to be convolved output by a multi-wavelength laser generation modulation module in the optical operation domain, realizing the forward propagation of a multi-layer neural network by increasing the parallel data flow, and obtaining the optical intensity results of each wavelength; using a photoelectric conversion module to output the optical intensity results of each received wavelength in the form of voltage signals; using an electrical processing and control module to perform post-processing on the received voltage signals to obtain the numerical results of the convolution operations, and controlling the multi-wavelength laser generation modulation module according to the numerical results and the required data obtained from the outside to update the data to be convolved input; using a data storage module to store the operation results of each layer of the forward propagation and transmit the data to be convolved required for the current parallel convolution operation.

[0014] The multi-layer optoelectronic convolution neural network system and method based on the optical parallel method provided by the present invention have at least the following technical effects:

[0015] By using a parallel optical convolution module to perform parallel convolution operations on N groups of multi-wavelength signals to be convolved received in the optical operation domain, realizing the forward propagation of a multi-layer neural network by increasing the parallel data flow, and based on such optical calculations, a large number of convolution operation operations are completed highly in parallel and then immediately followed by electrical calculations of storage, post-processing, and regulation, realizing high-speed parallel calculations of multiple layers, multiple convolution kernels, multiple channels, and multiple regions of the convolution neural network, greatly improving the training and inference efficiency of large-scale convolution operations. Description of the Drawings

[0016] Through the following description of the embodiments of the present invention with reference to the drawings, the above content and other objects, features, and advantages of the present invention will become clearer. In the drawings:

[0017] Figure 1 Schematically shows a schematic diagram of the structure of a multi-layer optoelectronic convolution neural network based on the optical parallel method provided by the embodiments of the present invention;

[0018] Figure 2 Schematically shows a schematic diagram of the structure of a multi-wavelength laser generation modulation module provided by the embodiments of the present invention;

[0019] Figure 3 Schematically shows a schematic diagram of the structure of a parallel optical convolution module provided by the embodiments of the present invention;

[0020] Figure 4 Schematically shows a schematic diagram of the structure of a photoelectric conversion module provided by the embodiments of the present invention;

[0021] Figure 5 Schematically shows a schematic diagram of the structure of an electrical processing and control module provided by the embodiments of the present invention;

[0022] Figure 6 Schematically shows a schematic diagram of a data storage module provided according to an embodiment of the present invention.

[0023] Reference numerals:

[0024] 1 - Multi - wavelength laser generation and modulation module; 11 - Multi - wavelength laser array; 12 - Modulator array; 13 - Wavelength division multiplexer array;

[0025] 2 - Parallel optical convolution module; 21 - Input optical circulator; 22 - Parallel optical convolution unit; 23 - Output optical circulator;

[0026] 3 - Photoelectric conversion module; 31 - Wavelength demultiplexer array; 32 - Photoelectric detector array; 33 - Trans - impedance amplifier array;

[0027] 4 - Electrical processing and control module; 41 - Electrical post - processing unit; 42 - First register 1; 43 - Discriminator; 44 - Second register 2; 45 - Modulator control unit;

[0028] 5 - Data storage module; 51 - Initial input storage unit; 52 - Intermediate - layer data storage unit; 53 - Terminal data storage unit. Detailed implementation manners

[0029] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. In the following detailed description, for the sake of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present invention. However, it is obvious that one or more embodiments can be implemented without these specific details. In addition, in the following description, descriptions of well - known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present invention.

[0030] The terms used herein are merely for describing specific embodiments and are not intended to limit the present invention. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0031] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0032] In the case of using expressions such as "at least one of A, B, and C, etc.", generally, it should be interpreted according to the meaning that those skilled in the art usually understand this expression (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0033] In the process of implementing the present invention, the applicant found that: As an emerging computing paradigm based on the characteristics of optical propagation, optical computing can transmit multiple signals in the same path and ensure extremely low crosstalk due to the extremely weak interaction between photons. Therefore, optical operations are suitable for high-parallel computing tasks, and it also has the advantages of high bandwidth, low transmission loss, low latency, and low power consumption, and is suitable for improving the training and inference efficiency of multi-layer convolutional neural networks. Based on this, the present invention utilizes the advantages of high parallelism, low crosstalk, and low loss of optical computing and combines the advantages of flexible storage and post-processing of traditional electrical computing to improve the information propagation efficiency of the convolutional neural network, thereby further improving the performance indicators of the convolutional neural network.

[0034] Figure 1 Schematically shows a schematic diagram of a multi-layer optoelectronic convolutional neural network structure based on an optical parallel method according to an embodiment of the present invention.

[0035] As Figure 1 shown, a multi-layer optoelectronic convolutional neural network system based on an optical parallel method may include:

[0036] A multi-wavelength laser generation and modulation module 1, configured to generate N groups of multi-wavelength laser signals representing N convolutional layers, perform information modulation and transmission path management on the N groups of multi-wavelength laser signals, so that the wavelengths in each group of multi-wavelength laser signals carry the data to be convolved in different convolutional regions of the same layer, and obtain N groups of multi-wavelength signals to be convolved, where N is a positive integer greater than 1.

[0037] A parallel optical convolution module 2, configured to perform parallel convolution operations on the received N groups of multi-wavelength signals to be convolved in the optical operation domain, and realize the forward propagation of the multi-layer neural network by increasing the parallel data flow rate, and obtain the optical intensity result of each wavelength.

[0038] An optoelectronic conversion module 3, configured to output the received optical intensity result of each wavelength in the form of a voltage signal.

[0039] An electrical processing and control module 4, configured to perform post-processing on the received voltage signal to obtain the numerical result of the convolution operation, and control the multi-wavelength laser generation and modulation module according to the numerical result and the required data obtained from the outside to update the input data to be convolved.

[0040] The data storage module 5 is configured to store the operation results of each layer in the forward propagation and transmit the data to be convolved required for the current parallel convolution operation.

[0041] It should be noted that Figure 1 The solid-line connections in [FIGURE] represent the propagation of light in optical interconnection structures such as optical fibers and optical waveguides; the dashed-line connections represent the electrical propagation of current or voltage.

[0042] Figure 2 Schematically shows the structural diagram of a multi-wavelength laser generation modulation module provided according to an embodiment of the present invention.

[0043] As Figure 2 shown, based on the above embodiment, the multi-wavelength laser generation modulation module 1 is sequentially connected to a multi-wavelength laser array 11, a modulator array 12, and a wavelength division multiplexer array 13.

[0044] The multi-wavelength laser array 11 includes N laser units for generating N groups of multi-wavelength laser signals representing N convolutional layers. Among them, each laser unit contains L1, L2,..., L N multi-wavelength lasers in number, for generating multi-wavelength laser signals carrying data in different regions in the Nth convolutional operation.

[0045] The modulator array 12 includes N modulator units for respectively modulating N groups of laser signals with different wavelengths and loading N groups of information onto the N groups of multi-wavelength laser signals, so that each wavelength of laser signal carries the corresponding data to be convolved. Among them, each modulator unit contains the same corresponding number of L1, L2,..., L N modulators in number.

[0046] The wavelength division multiplexer array 13 includes N wavelength division multiplexers for wavelength division multiplexing the data to be convolved in different operation regions of the same convolutional layer but represented by different wavelengths into the same transmission path to obtain N groups of multi-wavelength data to be convolved signals.

[0047] Among them, N is a positive integer greater than 1, and L1, L2,..., L N are all positive integers greater than or equal to 1.

[0048] It should be noted that Figure 2 the solid-line connections in [FIGURE] represent the propagation of laser signals in optical interconnection structures such as optical fibers and optical waveguides; the dashed-line connections represent the voltage signals for controlling the modulators.

[0049] Figure 3 Schematically shows the structural diagram of a parallel optical convolution module provided according to an embodiment of the present invention.

[0050] As Figure 3As shown, on the basis of the above embodiments, as a feasible way, the parallel optical convolution module 2 may include an input optical circulator 21, a parallel optical convolution unit 22, and an output optical circulator 23 that are connected in sequence.

[0051] The input optical circulator 21 includes a first optical circulator for separating the forward light input and the backward light output from each other on the channel to prevent the backward light output from entering the optical path of the forward light input.

[0052] The parallel optical convolution unit 22 is used to perform parallel operations on the data to be convolved in different regions of each convolution layer in the N-layer convolution layer.

[0053] The output optical circulator 23 includes a second optical circulator for separating the backward light input and the forward light output from each other on the channel to prevent the forward light output from entering the optical path of the backward light input.

[0054] A beam splitting device is further provided between the output of the multi-wavelength laser generation and modulation module 1 and the forward light input and the backward light input of the parallel optical convolution module 2. The beam splitting device is used to evenly divide the output of the multi-wavelength laser generation and modulation module 1 and input it into the forward light input port and the backward light input port of the parallel optical convolution module 2.

[0055] On the basis of the above embodiments, as another feasible way, the parallel optical convolution module 2 may include a parallel optical convolution unit 22. The parallel optical convolution unit 22 is used to perform parallel operations on the data to be convolved in different regions of each convolution layer in the N-layer convolution layer. That is to say, the parallel optical convolution module 2 may also only include the parallel optical convolution unit 22, and at the same time, the beam splitting device between the output of the multi-wavelength laser generation and modulation module and the parallel optical convolution module needs to be cancelled.

[0056] Furthermore, the input from the output of the multi-wavelength laser generation and modulation module 1 to the parallel optical convolution module 2 may be the same or different for each convolution layer in the N-layer convolution layer, including single forward light input, single backward light input, or forward and backward light input simultaneously.

[0057] Furthermore, the parallel optical convolution unit 22 may include: one of an optical convolution operator based on a Mach-Zehnder interferometer, an optical convolution operator based on a microring filter, an optical convolution operator based on a multimode interference coupler, an optical convolution operator based on diffractive optics, an optical convolution operator based on interference optics, an optical convolution operator based on wavelength division multiplexing, and an optical convolution operator based on a tunable optical amplifier and an optical attenuator.

[0058] It should be noted that Figure 3 The solid line connecting from left to right in represents the propagation process of the forward light from input to output; the dotted line connecting from right to left represents the propagation process of the backward light from input to output.

[0059] Figure 4 Schematically shows a schematic diagram of the optoelectronic conversion module provided according to an embodiment of the present invention.

[0060] As Figure 4 shown, on the basis of the above embodiment, the optoelectronic conversion module 3 includes a wavelength division multiplexer array 31, a photodetector array 32, and a transimpedance amplifier array 33 connected in sequence;

[0061] The wavelength division multiplexer array 31 includes N wavelength division multiplexers for separating the optical intensity results obtained by parallel operation from one channel into each individual channel;

[0062] The photodetector array 32 includes N groups of detector units, where each group of detector units respectively includes L1, L2,..., L N photodetectors in number, for performing optical intensity detection on the optical intensity results of each wavelength after convolution operation based on photoelectric conversion to convert the optical intensity result signal into a photocurrent signal;

[0063] The transimpedance amplifier array 33 includes N groups of transimpedance amplifier units, where each transimpedance amplifier unit respectively includes L1, L2,..., L N transimpedance amplifiers in number, for amplifying and converting the relatively small photocurrent signal output by the corresponding detector unit into a voltage signal.

[0064] It should be noted that Figure 4 the solid line connection in represents the propagation of the laser signal in optical interconnection structures such as optical fibers and optical waveguides; the dotted line connection represents the photocurrent signal obtained after passing through the photodetector; and the dashed line connection represents the amplified voltage signal.

[0065] Figure 5 Schematically shows a schematic diagram of the electrical processing and control module provided according to an embodiment of the present invention.

[0066] As Figure 5 shown, on the basis of the above embodiment, the electrical processing and control module 4 may include a first register 42 and an electrical post-processing unit 41, a first register 42, a discriminator 43, and a modulator control unit 45 connected in sequence, and the input of the first register 42 is connected to the output of the discriminator 43.

[0067] The electrical post-processing unit 41 is used for post-processing the input voltage signal, and obtaining the numerical value of the true convolution operation through electrical post-processing conversion operation.

[0068] The first register 42 is used to transfer the convolution operation data results that are not involved in the next convolution operation to the data storage module 5 after post - processing, and to retrieve the convolution - to - be data required for the next operation from the data storage module 5.

[0069] The discriminator 43 is used to implement the non - linear activation and max - pooling operations in the convolutional neural network, obtain the data after non - linear activation and pooling processing and the position information corresponding to the data change, and transmit them to the corresponding modules.

[0070] The second register 44 is used to record the position information where the non - linear activation and pooling operation data changes and transfer it to the data storage module 5.

[0071] The modulator control unit 45 is used to generate a corresponding regulation voltage according to the result obtained after a complete operation, and use this result as the initial input value for the next operation.

[0072] It should be noted that Figure 5 The unidirectional arrow represents the unidirectional flow of data; the bidirectional arrow represents the mutual exchange of data with the outside; the dashed line represents the transmission of data signals; and the dotted line represents the control signal.

[0073] Figure 6 Schematically shows the structural schematic diagram of the data storage module provided according to an embodiment of the present invention.

[0074] As Figure 6 shown, on the basis of the above - mentioned embodiment, the data storage module 5 may include:

[0075] The initial input storage unit 51 is used to store the convolution - to - be data of the initial input and send the corresponding data to the external module according to the calculation requirements.

[0076] The intermediate - layer data storage unit 52 is used to store the data generated during each complete operation process in the middle and the position information where the corresponding data changes, so as to extract the generated data and position information for numerical derivative operation for backpropagation, and send the corresponding data to the external module according to the calculation requirements.

[0077] The terminal data storage unit 53 is used to store the final data and the corresponding position information after the last - layer completion of convolution, non - linear activation, and pooling operations.

[0078] It should be noted that Figure 6 The unidirectional arrow represents the unidirectional flow of data; the bidirectional arrow represents the mutual exchange of data with the external module.

[0079] The multi-layer optoelectronic convolutional neural network system based on the optical parallel method provided by the present invention combines the advantages of high parallelism, low crosstalk, and low loss of optical computing with the advantages of flexible storage, post-processing, and regulation of electrical computing, and realizes high-speed parallel computing of multiple layers, multiple convolutional kernels, multiple channels, and multiple regions of a large-scale deep convolutional neural network with a large number of parameters. Therefore, it can further improve the training and inference efficiency of the convolutional neural network, so as to better meet the current increasingly complex deep learning task requirements related to the convolutional neural network.

[0080] Based on the above multi-layer optoelectronic convolutional neural network system based on the optical parallel method, an embodiment of the present invention further provides a parallel operation method, which may include:

[0081] Using the parallel optical convolution module 2 to perform parallel convolution operations on the N groups of multi-wavelength signals to be convolved output by the multi-wavelength laser generation modulation module 1 in the optical operation domain, and realizing the forward propagation of the multi-layer neural network by increasing the parallel data flow to obtain the light intensity results of each wavelength.

[0082] Using the optoelectronic conversion module 3 to output the light intensity results of each received wavelength in the form of voltage signals.

[0083] Using the electrical processing control module 4 to perform post-processing on the received voltage signals to obtain the numerical results of the convolution operations, and controlling the multi-wavelength laser generation modulation module to update the input data to be convolved according to the numerical results and the required data obtained from the outside.

[0084] Using the data storage module 5 to store the operation results of each layer of the forward propagation and the data to be convolved required for the current parallel convolution operation.

[0085] It should be noted that the implementation details and technical effects of the parallel computing method embodiment part are similar or the same as those of the system embodiment part. For specific details, please refer to the system embodiment part, and will not be elaborated here.

[0086] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functions, and operations of the possible implementations of systems and methods according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code, and the above-mentioned module, segment of a program, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and combinations of blocks in the block diagram or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0087] Those skilled in the art can understand that the features recited in the various embodiments and / or claims of the present invention can be combined or combined in various ways, even if such combinations or combinations are not explicitly recited in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recited in the various embodiments and / or claims of the present invention can be combined and combined in various ways. All such combinations and / or combinations fall within the scope of the present invention.

[0088] The above describes the embodiments of the present invention. However, these embodiments are merely for illustrative purposes and not for limiting the scope of the present invention. Although the embodiments are described separately above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. The scope of the present invention is defined by the appended claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present invention.

Claims

1. A multi-layer optoelectronic convolutional neural network system based on an optical parallel method, characterized in that Including: A multi-wavelength laser generation and modulation module (1), configured to generate N groups of multi-wavelength laser signals representing N convolutional layers, perform information modulation and transmission path management on the N groups of multi-wavelength laser signals, so that the wavelengths in each group of multi-wavelength laser signals carry the data to be convolved in different convolutional regions of the same layer, and obtain N groups of multi-wavelength signals to be convolved, where N is a positive integer greater than 1; A parallel optical convolution module (2), configured to perform parallel convolution operations on the data to be convolved in different regions of each convolutional layer in the received N groups of multi-wavelength signals to be convolved in the optical operation domain, and implement the forward propagation of the multi-layer neural network by increasing the parallel data flow, and obtain the optical intensity result of each wavelength; A photoelectric conversion module (3), configured to output the received optical intensity result of each wavelength in the form of a voltage signal; An electrical processing and control module (4), configured to perform post-processing on the received voltage signal to obtain the numerical result of the convolution operation, and control the multi-wavelength laser generation and modulation module according to the numerical result and the required data obtained from the outside to update the data to be convolved input; A data storage module (5), configured to store the operation results of each layer of forward propagation and transfer the data to be convolved required for the current parallel convolution operation.

2. The system according to claim 1, characterized in that, A multi-wavelength laser array (11), a modulator array (12), and a wavelength division multiplexer array (13) sequentially connected to the multi-wavelength laser generation and modulation module (1); The multi-wavelength laser array (11) includes N laser units for generating N groups of multi-wavelength laser signals representing N convolutional layers. Among them, each laser unit contains L1, L2, …, L N numbers of multi-wavelength lasers for generating multi-wavelength laser signals carrying data of different regions in the Nth convolutional operation; The modulator array (12) includes N modulator units for modulating N groups of laser signals with different wavelengths respectively, loading N groups of information onto the N groups of multi-wavelength laser signals respectively, so that each wavelength of laser signal carries the corresponding data to be convolved. Wherein, each modulator unit contains the same corresponding number of modulators of L1, L2, …, L N number of modulators; The wavelength division multiplexer array (13), which includes N wavelength division multiplexers, is used to wavelength-division multiplex the data to be convolved in different operation regions of the same convolutional layer but represented by different wavelengths into the same transmission path to obtain N groups of multi-wavelength signals to be convolved; where N is a positive integer greater than 1, and L1, L2, …, L N are all positive integers greater than or equal to 1.

3. The system according to claim 1 or 2, characterized in that, The parallel optical convolution module (2) includes an input optical circulator (21), a parallel optical convolution unit (22), and an output optical circulator (23) connected in sequence; The input optical circulator (21) includes a first optical circulator, which is used to separate the forward light input and the backward light output on the channel to prevent the backward light output from entering the optical path of the forward light input; The parallel optical convolution unit (22) is used to implement the parallel operation of the data to be convolved in different regions of each convolutional layer in the N convolutional layers; The output optical circulator (23) includes a second optical circulator, which is used to separate the backward light input and the forward light output on the channel to prevent the forward light output from entering the optical path of the backward light input; A beam splitting device is further provided between the forward light input and the backward light input from the output of the multi-wavelength laser generation and modulation module (1) to the parallel optical convolution module (2), and the beam splitting device is used to evenly divide the output of the multi-wavelength laser generation and modulation module (1) and input it into the forward light input port and the backward light input port of the parallel optical convolution module (2).

4. The system according to claim 1 or 2, characterized in that, The parallel optical convolution module (2) includes a parallel optical convolution unit (22); The parallel optical convolution unit (22) is used to implement the parallel operation of the data to be convolved in different regions of each convolutional layer in the N convolutional layers.

5. The system according to claim 3, characterized in that, The output of the multi-wavelength laser generation modulation module (1) to the input of the parallel optical convolution module (2) is the same or different for each convolutional layer in the N-layer convolutional layer, including single forward optical input, single reverse optical input, or simultaneous forward and reverse input.

6. The system according to claim 4, characterized in that, The parallel optical convolution unit (22) includes: one of an optical convolution operator based on a Mach-Zehnder interferometer, an optical convolution operator based on a microring filter, an optical convolution operator based on a multimode interference coupler, an optical convolution operator based on diffractive optics, an optical convolution operator based on interference optics, an optical convolution operator based on wavelength division multiplexing, and an optical convolution operator based on a tunable optical amplifier and an optical attenuator.

7. The system according to claim 1 or 2, characterized in that, The optoelectronic conversion module (3) includes a wavelength division demultiplexer array (31), a photodetector array (32), and a transimpedance amplifier array (33) connected in sequence. The wavelength division demultiplexer array (31) includes N wavelength division demultiplexers for separating the optical intensity results obtained from parallel operations from one channel into each individual channel. The photodetector array (32) includes N groups of detector units, where each group of detector units respectively includes L1, L2, …, L N photodetectors in number, which are used to detect the light intensity result of each wavelength after convolution operation based on photoelectric conversion so as to convert the light intensity result number into a photocurrent signal; Transimpedance amplifier array (33), including N groups of transimpedance amplifier units, where each transimpedance amplifier unit respectively includes L1, L2, …, L N numbers of transimpedance amplifiers for amplifying the photocurrent signals output by corresponding detector units and converting them into voltage signals.

8. The system according to claim 1 or 2, characterized in that, The electrical processing and control module (4) includes a first register (42) and an electrical post-processing unit (41), a first register (42), a discriminator (43), and a modulator control unit (45) connected in sequence. The input of the first register (42) is connected to the output of the discriminator (43). The electrical post-processing unit (41) is used to perform post-processing on the input voltage signal and obtain the numerical value of the convolution operation through electrical post-processing transformation operations. The first register (42) is used to transfer the convolution operation data results that are not involved in the next convolution operation after post-processing to the data storage module (5), and retrieve the convolution data required for the next operation from the data storage module (5). The discriminator (43) is used to implement the non-linear activation and max pooling operations in the convolutional neural network to obtain the data after non-linear activation and pooling processing and the position information corresponding to the data change. The second register (44) is used to record the position information where the non-linear activation and pooling operation data changes and transfer it to the data storage module (5). The modulator control unit (45) is used to generate a corresponding control voltage based on the result obtained after a complete operation and use this result as the initial input value for the next operation.

9. The system according to claim 1 or 2, characterized in that The data storage module (5) includes: An initial input storage unit (51) for storing the initial input convolution data and sending corresponding data to external modules according to calculation requirements. An intermediate layer data storage unit (52) for storing the data generated during each complete operation in the middle and the position information corresponding to the data change, for subsequent extraction of the generated data and position information for numerical derivative operations for backpropagation, and sending corresponding data to external modules according to calculation requirements. A terminal data storage unit (53) for storing the final data and corresponding position information after the last layer completes convolution, non-linear activation, and pooling operations.

10. A parallel computing method, characterized in that, The method is implemented based on the system according to any one of claims 1-9 and includes: Use the parallel optical convolution module (2) to perform parallel convolution operations on the data to be convolved in different regions of each convolutional layer in the N groups of multi-wavelength signals to be convolved output by the received multi-wavelength laser generation modulation module (1) in the optical operation domain, and realize the forward propagation of the multi-layer neural network by increasing the parallel data flow to obtain the light intensity results of each wavelength; Use the optoelectronic conversion module (3) to output the received light intensity results of each wavelength in the form of voltage signals; Use the electrical processing and control module (4) to perform post-processing on the received voltage signals to obtain the numerical results of the convolution operations, and control the multi-wavelength laser generation modulation module according to the numerical results and the required data obtained from the outside to update the data to be convolved input; Use the data storage module (5) to store the operation results of each layer of forward propagation and the data to be convolved required for the current parallel convolution operation.

Images