Multilayer photoelectric convolutional neural network system and method based on optical parallel method

Abstract

The invention provides a multilayer photoelectric convolutional neural network system and method based on an optical parallel method, and relates to the field of photoelectric neural network calculation, and the system comprises a multi-wavelength laser generation and modulation module which is used for generating N groups of multi-wavelength laser signals representing N convolutional layers and carrying out information modulation and transmission path management, enabling the wavelength of each group of multi-wavelength laser signals to carry data to be convolved of different convolution areas on the same layer; the parallel optical convolution module is used for carrying out parallel convolution on the N groups of multi-wavelength signals to be convolved in an optical operation domain and realizing forward propagation of a multi-layer network by increasing parallel traffic of data; the photoelectric conversion module is used for converting the light intensity result of each wavelength obtained by convolution into a voltage signal; the electric processing control module is used for post-processing the voltage signal and updating the data to be convolved according to a post-processed numerical result and the acquired required data; and the data storage module is used for storing an operation result of each layer of forward propagation and transmitting to-be-convolved data of the current parallel convolution operation.

Description

Technical Field

[0001] The present invention relates to the field of optoelectronic neural network computing technology, and in particular to a multi-layer optoelectronic convolutional neural network system and method based on an optical parallel method. Background Art

[0002] As one of the representative algorithms in current deep learning technology, convolutional neural networks are widely used in tasks such as target detection and image and video recognition. They use convolution operations to achieve feature extraction and local connection, nonlinear activation to increase the nonlinear representation ability of the network, and pooling to reduce the risk of overfitting by compressing parameters. At present, with the continuous development of deep learning technology and the increasing complexity of data tasks, the parameter scale and number of layers of convolutional neural networks are constantly expanding, which brings huge challenges to the training and reasoning efficiency of convolutional neural networks. Summary of the invention

[0003] In view of the above problems, the present invention provides a multi-layer optoelectronic convolutional neural network system and method based on an optical parallel method, which is used to at least partially solve the above technical problems.

[0004] According to a first aspect of an embodiment of the present invention, a multi-layer optoelectronic convolutional neural network system based on an optical parallel method is provided, comprising: a multi-wavelength laser generation and modulation module, configured to generate N groups of multi-wavelength laser signals representing N layers of convolutional layers, and to perform information modulation and transmission path management on the N groups of multi-wavelength laser signals, so that the wavelengths in each group of multi-wavelength laser signals carry data to be convolved in different convolutional regions of the same layer, and obtain N groups of multi-wavelength signals to be convolved, where N is a positive integer greater than 1; a parallel optical convolution module, configured to perform parallel convolution operations on the received N groups of multi-wavelength signals to be convolved in an optical operation domain. The multi-wavelength laser generation and modulation module is configured to update the input data to be convolved according to the numerical result and the required data obtained from the outside; the data storage module is configured to store the operation results of each layer of the forward propagation and transmit the data to be convolved required for the current parallel convolution operation.

[0005] According to an embodiment of the present invention, a multi-wavelength laser array, a modulator array, and a wavelength division multiplexer array are connected in sequence to the multi-wavelength laser generation modulation module; the multi-wavelength laser array includes N laser units for generating N groups of multi-wavelength laser signals representing N layers of convolutional layers, wherein each laser unit includes L1, L2, ..., L NA number of multi-wavelength lasers are used to generate multi-wavelength laser signals carrying data of different regions in the N-th layer of convolution operation; a modulator array includes N modulator units, which are used to modulate N groups of laser signals of different wavelengths respectively, and load N groups of information onto the N groups of multi-wavelength laser signals respectively, so that each wavelength of the laser signal carries the corresponding data to be convolved, wherein each modulator unit contains the same number of L1, L2, ..., L as the laser unit. N modulators of a certain number; a wavelength division multiplexer array, comprising N wavelength division multiplexers, for wavelength division multiplexing the to-be-convolved data of different operation areas of the same convolution layer but represented by different wavelengths into the same transmission path, thereby obtaining N groups of multi-wavelength to-be-convolved signals; wherein N is a positive integer greater than 1, L1, L2, ..., L N are all positive integers greater than or equal to 1.

[0006] According to an embodiment of the present invention, a parallel optical convolution module includes an input optical circulator, a parallel optical convolution unit and an output optical circulator connected in sequence; the input optical circulator includes a first optical circulator for separating the forward light input and the reverse light output on the channel to prevent the reverse light output from entering the optical path of the forward light input; the parallel optical convolution unit is used to realize parallel operation of the data to be convolved in different areas of each convolution layer in N layers of convolution layers; the output optical circulator includes a second optical circulator for separating the reverse light input and the forward light output on the channel to prevent the forward light output from entering the optical path of the reverse light input; a beam splitter is also provided between the output of the multi-wavelength laser generation and modulation module and the forward light input and reverse light input of the parallel optical convolution module, and the beam splitter is used to evenly divide the output of the multi-wavelength laser generation and modulation module and input it into the forward light input port and the reverse light input port of the parallel optical convolution module.

[0007] According to an embodiment of the present invention, the parallel optical convolution module includes a parallel optical convolution unit; the parallel optical convolution unit is used to realize parallel calculation of the data to be convolved in different areas of each convolution layer in N convolution layers.

[0008] According to an embodiment of the present invention, the output of the multi-wavelength laser generation modulation module to the input of the parallel optical convolution module is the same or different for each convolution layer in the N convolution layers, including a single forward light input, a single reverse light input, or simultaneous forward and reverse input.

[0009] According to an embodiment of the present invention, the parallel optical convolution unit includes: an optical convolution operator based on a Mach-Zehnder interferometer, an optical convolution operator based on a microring filter, an optical convolution operator based on a multimode interference coupler, a convolution operator based on diffraction optics, a convolution operator based on interference optics, an optical convolution operator based on wavelength division multiplexing, and one of the optical convolution operators based on an adjustable optical amplifier and an optical attenuator.

[0010] According to an embodiment of the present invention, the photoelectric conversion module includes a wavelength division multiplexer array, a photodetector array and a transimpedance amplifier array connected in sequence; the wavelength division multiplexer array includes N wavelength division multiplexers, which are used to separate the light intensity results obtained by parallel operation from one channel into each separate channel; the photodetector array includes N groups of detector units, wherein each group of detector units includes L1, L2, ..., L N A number of photodetectors are used to detect the light intensity result of each wavelength after the convolution operation based on photoelectric conversion, so as to convert the light intensity result signal into a photocurrent signal; a transimpedance amplifier array includes N groups of transimpedance amplifier units, wherein each transimpedance amplifier unit includes L1, L2, ..., L N A number of transimpedance amplifiers are used to amplify the photocurrent signal output by the corresponding detector unit and convert it into a voltage signal.

[0011] According to an embodiment of the present invention, the electrical processing control module includes a first register and an electrical post-processing unit, a first register, a discriminator, and a modulator control unit connected in sequence, wherein the input of the first register is connected to the output of the discriminator; the electrical post-processing unit is used to post-process the input voltage signal, and obtain the value of the convolution operation through an electrical post-processing conversion operation; the first register is used to transfer the post-processed convolution operation data results of the part not involved in the next convolution operation to the data storage module, and to retrieve the data to be convolved required for the next operation from the data storage module; the discriminator is used to implement the nonlinear activation and maximum pooling operations in the convolutional neural network, and obtain the data after nonlinear activation and pooling processing and the position information corresponding to the data change; the second register is used to record the position information of the change of the nonlinear activation and pooling operation data and transfer it to the data storage module; the modulator control unit is used to generate a corresponding control voltage according to the result obtained after a complete operation, and use the result as the initial input value for the next operation.

[0012] According to an embodiment of the present invention, the data storage module includes: an initial input storage unit, which is used to store the initial input data to be convolved, and send corresponding data to an external module according to computing requirements; an intermediate layer data storage unit, which is used to store the data generated during each complete operation and the position information of the corresponding data changes, so as to facilitate the subsequent extraction of the generated data and position information for numerical derivative operations and back propagation, and send corresponding data to an external module according to computing requirements; a final data storage unit, which is used to store the final data and corresponding position information of the last layer that completes convolution, nonlinear activation, and pooling operations.

[0013] The second aspect of an embodiment of the present invention provides a parallel computing method, which is implemented based on the above-mentioned system, including: using a parallel optical convolution module to perform parallel convolution operations on N groups of multi-wavelength signals to be convolved output by a received multi-wavelength laser generation modulation module in an optical operation domain, and realizing the forward propagation of a multi-layer neural network by increasing the parallel flow of data to obtain the light intensity result of each wavelength; using a photoelectric conversion module to output the received light intensity result of each wavelength in the form of a voltage signal; using an electrical processing control module to post-process the received voltage signal to obtain a numerical result of the convolution operation, and controlling the multi-wavelength laser generation modulation module according to the numerical result and the required data obtained from the outside to update the input data to be convolved; using a data storage module to store the operation results of each layer of the forward propagation and transfer the data to be convolved required for the current parallel convolution operation.

[0014] The multi-layer optoelectronic convolutional neural network system and method based on the optical parallel method provided by the present invention have at least the following technical effects:

[0015] Through the parallel optical convolution module, parallel convolution operations are performed on the received N groups of multi-wavelength signals to be convolved in the optical operation domain. The forward propagation of the multi-layer neural network is realized by increasing the parallel flow of data. Based on this highly parallel optical calculation, a large number of convolution operations are completed, followed by the storage, post-processing and regulation of electrical calculations, to achieve high-speed parallel calculation of multiple layers, multiple convolution kernels, multiple channels and multiple regions of the convolutional neural network, which greatly improves the training and reasoning efficiency of large-scale convolution operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0017] Figure 1 A schematic diagram of a multi-layer optoelectronic convolutional neural network structure based on an optical parallel method provided according to an embodiment of the present invention is schematically shown;

[0018] Figure 2 The structure diagram of the multi-wavelength laser generation and modulation module provided in accordance with an embodiment of the present invention is schematically shown;

[0019] Figure 3 The schematic diagram shows the structure of a parallel optical convolution module provided according to an embodiment of the present invention;

[0020] Figure 4 The structure diagram of the photoelectric conversion module provided according to the embodiment of the present invention is schematically shown;

[0021] Figure 5 A schematic diagram of the structure of an electric processing control module provided according to an embodiment of the present invention is shown;

[0022] Figure 6 The structure diagram of the data storage module provided in the embodiment of the present invention is schematically shown.

[0023] Reference numerals:

[0024] 1-multi-wavelength laser generation modulation module; 11-multi-wavelength laser array; 12-modulator array; 13-wavelength division multiplexer array;

[0025] 2-parallel optical convolution module; 21-input optical circulator; 22-parallel optical convolution unit; 23-output optical circulator;

[0026] 3-photoelectric conversion module; 31-wavelength division multiplexer array; 32-photodetector array; 33-transimpedance amplifier array;

[0027] 4-electrical processing control module; 41-electrical post-processing unit; 42-first register 1; 43-discriminator; 44-second register 2; 45-modulator control unit;

[0028] 5-data storage module; 51-initial input storage unit; 52-intermediate data storage unit; 53-final data storage unit. DETAILED DESCRIPTION

[0029] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0030] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0031] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0032] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0033] In the process of realizing the present invention, the applicant found that optical computing, as an emerging computing paradigm based on the characteristics of optical propagation, can propagate multiple signals in the same path and ensure extremely low crosstalk due to the extremely weak interaction between photons. Therefore, optical computing is suitable for high-parallel computing tasks, and it also has the advantages of high bandwidth, low transmission loss, low latency and low power consumption, and is suitable for improving the training and reasoning efficiency of multi-layer convolutional neural networks. Based on this, the present invention utilizes the advantages of high parallelism, low crosstalk and low loss of optical computing and combines the advantages of flexible storage and post-processing of traditional electrical computing to improve the information propagation efficiency of convolutional neural networks, thereby further improving the performance indicators of convolutional neural networks.

[0034] Figure 1 A schematic diagram of a multi-layer optoelectronic convolutional neural network structure based on an optical parallel method provided according to an embodiment of the present invention is schematically shown.

[0035] like Figure 1 As shown, a multi-layer optoelectronic convolutional neural network system based on an optical parallel method may include:

[0036] The multi-wavelength laser generation and modulation module 1 is configured to generate N groups of multi-wavelength laser signals representing N convolution layers, perform information modulation and transmission path management on the N groups of multi-wavelength laser signals, so that the wavelength in each group of multi-wavelength laser signals carries the data to be convolved in different convolution areas of the same layer, and obtain N groups of multi-wavelength signals to be convolved, where N is a positive integer greater than 1.

[0037] The parallel optical convolution module 2 is configured to perform parallel convolution operations on the received N groups of multi-wavelength signals to be convolved in the optical operation domain, and realize the forward propagation of the multi-layer neural network by increasing the parallel flow of data to obtain the light intensity result of each wavelength.

[0038] The photoelectric conversion module 3 is configured to output the received light intensity result of each wavelength in the form of a voltage signal.

[0039] The electrical processing control module 4 is configured to post-process the received voltage signal to obtain a numerical result of the convolution operation, and to control the multi-wavelength laser generation modulation module to update the input data to be convolved according to the numerical result and required data obtained from the outside.

[0040] The data storage module 5 is configured to store the operation results of each layer of the forward propagation and to transfer the convolution data required for the current parallel convolution operation.

[0041] It should be noted that Figure 1 The solid line connections in the figure represent the propagation of light in optical interconnect structures such as optical fibers and optical waveguides; the dotted line connections represent the electrical propagation of current or voltage.

[0042] Figure 2 The structure diagram of the multi-wavelength laser generation and modulation module provided according to an embodiment of the present invention is schematically shown.

[0043] like Figure 2 As shown, based on the above embodiment, the multi-wavelength laser generating and modulating module 1 is sequentially connected to a multi-wavelength laser array 11 , a modulator array 12 , and a wavelength division multiplexer array 13 .

[0044] The multi-wavelength laser array 11 includes N laser units, which are used to generate N groups of multi-wavelength laser signals representing N convolutional layers, wherein each laser unit includes L1, L2, ..., L N A number of multi-wavelength lasers are used to generate multi-wavelength laser signals carrying data of different regions in the Nth layer of convolution operation.

[0045] The modulator array 12 includes N modulator units, which are used to modulate N groups of laser signals with different wavelengths respectively, and load N groups of information onto N groups of multi-wavelength laser signals respectively, so that each wavelength of the laser signal carries the corresponding data to be convolved, wherein each modulator unit includes the same number of L1, L2, ..., L N number of modulators.

[0046] The wavelength division multiplexer array 13 includes N wavelength division multiplexers, which are used to wavelength division multiplex the data to be convolved in different operation areas of the same convolution layer but represented by different wavelengths into the same transmission path to obtain N groups of multi-wavelength signals to be convolved.

[0047] Where N is a positive integer greater than 1, L1, L2, ..., L N are all positive integers greater than or equal to 1.

[0048] It should be noted that Figure 2 The solid line connection in represents the propagation of laser signals in optical interconnect structures such as optical fibers and optical waveguides; the dotted line connection represents the voltage signal that controls the modulator.

[0049] Figure 3 The schematic diagram shows the structure of a parallel optical convolution module provided according to an embodiment of the present invention.

[0050] like Figure 3As shown, based on the above embodiment, as a feasible manner, the parallel optical convolution module 2 may include an input optical circulator 21, a parallel optical convolution unit 22 and an output optical circulator 23 which are connected in sequence.

[0051] The input optical circulator 21 includes a first optical circulator for separating the forward optical input and the reverse optical output from each other in channels to prevent the reverse optical output from entering the optical path of the forward optical input.

[0052] The parallel optical convolution unit 22 is used to implement parallel operations on the data to be convolved in different areas of each convolution layer in the N convolution layers.

[0053] The output optical circulator 23 includes a second optical circulator for separating the reverse optical input and the forward optical output from each other on the channel to prevent the forward optical output from entering the optical path of the reverse optical input.

[0054] A beam splitter is also provided between the output of the multi-wavelength laser generation and modulation module 1 and the forward light input and reverse light input of the parallel optical convolution module 2. The beam splitter is used to evenly divide the output of the multi-wavelength laser generation and modulation module 1 and input them into the forward light input port and reverse light input port of the parallel optical convolution module 2.

[0055] On the basis of the above embodiment, as another feasible way, the parallel optical convolution module 2 may include a parallel optical convolution unit 22. The parallel optical convolution unit 22 is used to realize the parallel operation of the data to be convolved in different regions of each convolution layer in the N convolution layers. That is, the parallel optical convolution module 2 may also only include the parallel optical convolution unit 22, and at the same time, it is necessary to cancel the beam splitting device between the output of the multi-wavelength laser generation modulation module and the parallel optical convolution module.

[0056] Furthermore, the output of the multi-wavelength laser generation modulation module 1 to the input of the parallel optical convolution module 2 is the same or different for each convolution layer in the N convolution layers, including single forward light input, single reverse light input or simultaneous forward and reverse input.

[0057] Furthermore, the parallel optical convolution unit 22 may include: an optical convolution operator based on a Mach-Zehnder interferometer, an optical convolution operator based on a microring filter, an optical convolution operator based on a multimode interference coupler, a convolution operator based on diffraction optics, a convolution operator based on interference optics, an optical convolution operator based on wavelength division multiplexing, and an optical convolution operator based on an adjustable optical amplifier or an optical attenuator.

[0058] It should be noted that Figure 3 The solid lines in the figure connect from left to right to represent the propagation process of the forward light from input to output; the dotted lines connect from right to left to represent the propagation process of the reverse light from input to output.

[0059] Figure 4 The schematic diagram shows the structure of the photoelectric conversion module provided according to the embodiment of the present invention.

[0060] like Figure 4 As shown, based on the above embodiment, the photoelectric conversion module 3 includes a wavelength division multiplexer array 31, a photodetector array 32 and a transimpedance amplifier array 33 which are connected in sequence;

[0061] The wavelength division multiplexer array 31 includes N wavelength division multiplexers, which are used to separate the light intensity results obtained by parallel operation from one channel into each individual channel;

[0062] The photodetector array 32 includes N groups of detector units, wherein each group of detector units includes L1, L2, ..., L N A number of photodetectors are used to detect the light intensity result of each wavelength after the convolution operation based on photoelectric conversion, so as to convert the light intensity result signal into a photocurrent signal;

[0063] The transimpedance amplifier array 33 includes N groups of transimpedance amplifier units, wherein each transimpedance amplifier unit includes L1, L2, ..., L N A number of transimpedance amplifiers are used to amplify the smaller photocurrent signal output by the corresponding detector unit and convert it into a voltage signal.

[0064] It should be noted that Figure 4 The solid line connection in the figure represents the propagation of the laser signal in optical interconnection structures such as optical fibers and optical waveguides; the dotted line connection represents the photocurrent signal obtained after passing through the photodetector; and the dotted line connection represents the amplified voltage signal.

[0065] Figure 5 The schematic diagram shows the structure of an electric processing control module provided according to an embodiment of the present invention.

[0066] like Figure 5 As shown, based on the above embodiment, the electrical processing control module 4 may include a first register 42 and an electrical post-processing unit 41, a first register 42, a discriminator 43, and a modulator control unit 45 connected in sequence, and the input of the first register 42 is connected to the output of the discriminator 43.

[0067] The electrical post-processing unit 41 is used to perform post-processing on the input voltage signal, and obtain the actual value of the convolution operation through electrical post-processing conversion operation.

[0068] The first register 42 is used to transfer the post-processed convolution operation data result that is not involved in the next convolution operation to the data storage module 5, and to retrieve the to-be-convolved data required for the next operation from the data storage module 5.

[0069] The discriminator 43 is used to implement the nonlinear activation and maximum pooling operations in the convolutional neural network, obtain the data after nonlinear activation and pooling processing and the position information corresponding to the data change, and transmit them to the corresponding module.

[0070] The second register 44 is used to record the position information of the changes in the nonlinear activation and pooling operation data and transfer it to the data storage module 5.

[0071] The modulator control unit 45 is used to generate a corresponding control voltage according to the result obtained after a complete operation, and use the result as the initial input value for the next operation.

[0072] It should be noted that Figure 5 The one-way arrow represents the one-way flow of data; the two-way arrow represents the exchange of data with the outside world; the dotted line represents the transmission of data signals; and the dotted line represents the control signal.

[0073] Figure 6 The structure diagram of the data storage module provided in the embodiment of the present invention is schematically shown.

[0074] like Figure 6 As shown, based on the above embodiment, the data storage module 5 may include:

[0075] The initial input storage unit 51 is used to store the initial input data to be convolved and send the corresponding data to the external module according to the calculation requirements.

[0076] The intermediate layer data storage unit 52 is used to store the data generated during each complete operation and the position information of the corresponding data changes, so as to facilitate the subsequent extraction of the generated data and position information for numerical derivative operations and reverse propagation, and to send corresponding data to external modules according to computing requirements.

[0077] The final data storage unit 53 is used to store the final data and corresponding position information of the last layer that completes the convolution, nonlinear activation, and pooling operations.

[0078] It should be noted that Figure 6 The one-way arrows represent the one-way flow of data; the two-way arrows represent the exchange of data with external modules.

[0079] The multi-layer optoelectronic convolutional neural network system based on the optical parallel method provided by the present invention combines the advantages of high parallelism, low crosstalk and low loss of optical computing with the advantages of flexible storage, post-processing and regulation of electrical computing, so as to realize high-speed parallel computing of multiple layers, multiple convolution kernels, multiple channels and multiple regions of large-scale deep convolutional neural networks with a large number of parameters, thereby further improving the training and reasoning efficiency of convolutional neural networks, thereby better meeting the increasingly complex deep learning task requirements of convolutional neural networks.

[0080] Based on the above-mentioned multi-layer optoelectronic convolutional neural network system based on the optical parallel method, an embodiment of the present invention further provides a parallel computing method, which may include:

[0081] The parallel optical convolution module 2 is used to perform parallel convolution operations on the N groups of multi-wavelength signals to be convolved output by the received multi-wavelength laser generation modulation module 1 in the optical operation domain. By increasing the parallel flow of data, the forward propagation of the multi-layer neural network is realized to obtain the light intensity result of each wavelength.

[0082] The photoelectric conversion module 3 is used to output the received light intensity result of each wavelength in the form of a voltage signal.

[0083] The electrical processing control module 4 is used to post-process the received voltage signal to obtain the numerical result of the convolution operation, and the multi-wavelength laser generation modulation module is controlled according to the numerical result and the required data obtained from the outside to update the input data to be convolved.

[0084] The data storage module 5 is used to store the operation results of each layer of the forward propagation and to transfer the convolution data required for the current parallel convolution operation.

[0085] It should be noted that the implementation details and technical effects of the parallel computing method embodiment part and the system embodiment part are similar or identical to each other. For specific details, please refer to the system embodiment part, which will not be repeated here.

[0086] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems and methods according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment or a part of a code, and the above-mentioned module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0087] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All of these combinations and / or combinations fall within the scope of the present invention.

[0088] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination. The scope of the present invention is defined by the attached claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A multi-layer optoelectronic convolutional neural network system based on an optical parallel method, characterized in that: include: The multi-wavelength laser generation and modulation module (1) is configured to generate N groups of multi-wavelength laser signals representing N convolution layers, perform information modulation and transmission path management on the N groups of multi-wavelength laser signals, so that the wavelengths in each group of multi-wavelength laser signals carry data to be convolved in different convolution areas of the same layer, and obtain N groups of multi-wavelength signals to be convolved, where N is a positive integer greater than 1; The parallel optical convolution module (2) is configured to perform parallel convolution operations on the received N groups of multi-wavelength signals to be convolved in the optical operation domain, and realize the forward propagation of the multi-layer neural network by increasing the parallel flow of data, thereby obtaining the light intensity result of each wavelength; The photoelectric conversion module (3) is configured to output the received light intensity result of each wavelength in the form of a voltage signal; An electrical processing control module (4) is configured to perform post-processing on the received voltage signal to obtain a numerical result of the convolution operation, and to control the multi-wavelength laser generation and modulation module according to the numerical result and required data obtained from the outside to update the input data to be convolved; The data storage module (5) is configured to store the operation results of each layer of the forward propagation and to transfer the convolution data required for the current parallel convolution operation.

2. The system according to claim 1, characterized in that The multi-wavelength laser generation and modulation module (1) is connected in sequence to a multi-wavelength laser array (11), a modulator array (12), and a wavelength division multiplexer array (13); The multi-wavelength laser array (11) comprises N laser units, which are used to generate N groups of multi-wavelength laser signals representing N convolutional layers, wherein each laser unit comprises L1, L2, ..., L N A number of multi-wavelength lasers are used to generate multi-wavelength laser signals carrying data of different regions in the Nth layer of convolution operation; A modulator array (12) includes N modulator units, which are used to modulate N groups of laser signals with different wavelengths respectively, and load the N groups of information onto the N groups of multi-wavelength laser signals respectively, so that each wavelength of the laser signal carries corresponding data to be convolved, wherein each modulator unit includes the same number of L1, L2, ..., L N number of modulators; A wavelength division multiplexer array (13), comprising N wavelength division multiplexers, for wavelength division multiplexing the to-be-convolved data of different operation areas of the same convolution layer but represented by different wavelengths into the same transmission path, thereby obtaining N groups of multi-wavelength to-be-convolved signals; Where N is a positive integer greater than 1, L1, L2, ..., L N are all positive integers greater than or equal to 1.

3. The system according to claim 1 or 2, characterized in that: The parallel optical convolution module (2) comprises an input optical circulator (21), a parallel optical convolution unit (22) and an output optical circulator (23) which are connected in sequence; The input optical circulator (21) comprises a first optical circulator, which is used to separate the forward optical input and the reverse optical output from each other on the channel, so as to prevent the reverse optical output from entering the optical path of the forward optical input; The parallel optical convolution unit (22) is used to realize parallel operation of the data to be convolved in different areas of each convolution layer in N convolution layers; The output optical circulator (23) comprises a second optical circulator, which is used to separate the reverse optical input and the forward optical output from each other on the channel, so as to prevent the forward optical output from entering the optical path of the reverse optical input; A beam splitter is also provided between the output of the multi-wavelength laser generation and modulation module (1) and the forward light input and reverse light input of the parallel optical convolution module (2), and the beam splitter is used to evenly divide the output of the multi-wavelength laser generation and modulation module (1) and input them into the forward light input port and reverse light input port of the parallel optical convolution module (2).

4. The system according to claim 1 or 2, characterized in that: The parallel optical convolution module (2) comprises a parallel optical convolution unit (22); The parallel optical convolution unit (22) is used to realize parallel computing of the data to be convolved in different regions of each convolution layer in N convolution layers.

5. The system according to claim 3, characterized in that The input from the output of the multi-wavelength laser generation modulation module (1) to the parallel optical convolution module (2) is the same or different for each convolution layer in the N convolution layers, including single forward light input, single reverse light input or simultaneous forward and reverse input.

6. The system according to claim 4, characterized in that The parallel optical convolution unit (22) comprises: an optical convolution operator based on a Mach-Zehnder interferometer, an optical convolution operator based on a micro-ring filter, an optical convolution operator based on a multi-mode interference coupler, a convolution operator based on diffraction optics, a convolution operator based on interference optics, an optical convolution operator based on wavelength division multiplexing, and an optical convolution operator based on an adjustable optical amplifier or an optical attenuator.

7. The system according to claim 1 or 2, characterized in that: The photoelectric conversion module (3) comprises a wavelength division multiplexer array (31), a photodetector array (32) and a transimpedance amplifier array (33) which are connected in sequence; The wavelength division multiplexer array (31) comprises N wavelength division multiplexers, which are used to separate the light intensity results obtained by parallel operation from one channel into each individual channel; The photoelectric detector array (32) comprises N groups of detector units, wherein each group of detector units comprises L1, L2, ..., L N A number of photodetectors are used to detect the light intensity result of each wavelength after the convolution operation based on photoelectric conversion, so as to convert the light intensity result signal into a photocurrent signal; The transimpedance amplifier array (33) comprises N groups of transimpedance amplifier units, wherein each transimpedance amplifier unit comprises L1, L2, ..., L N A number of transimpedance amplifiers are used to amplify the photocurrent signal output by the corresponding detector unit and convert it into a voltage signal.

8. The system according to claim 1 or 2, characterized in that: The electrical processing control module (4) comprises a first register (42) and an electrical post-processing unit (41), a first register (42), a discriminator (43), and a modulator control unit (45) connected in sequence, wherein an input of the first register (42) is connected to an output of the discriminator (43); The electrical post-processing unit (41) is used to post-process the input voltage signal, and obtain the value of the convolution operation through the electrical post-processing conversion operation; The first register (42) is used to transfer the post-processed convolution operation data result of the part not involved in the next convolution operation to the data storage module (5), and to retrieve the to-be-convolved data required for the next operation from the data storage module (5); A discriminator (43) is used to implement nonlinear activation and maximum pooling operations in a convolutional neural network, and obtain data after nonlinear activation and pooling processing and position information corresponding to data changes; A second register (44) is used to record position information of changes in nonlinear activation and pooling operation data and transfer the information to the data storage module (5); The modulator control unit (45) is used to generate a corresponding control voltage according to a result obtained after a complete operation, and use the result as an initial input value for the next operation.

9. The system according to claim 1 or 2, characterized in that: The data storage module (5) comprises: An initial input storage unit (51), used to store the initial input data to be convolved, and to send the corresponding data to the external module according to the calculation requirements; The intermediate layer data storage unit (52) is used to store the data generated during each complete operation and the position information of the corresponding data changes, so as to facilitate the subsequent extraction of the generated data and position information for numerical derivation operation and reverse propagation, and to send the corresponding data to the external module according to the calculation requirements; The final data storage unit (53) is used to store the final data and corresponding position information of the last layer that completes the convolution, nonlinear activation, and pooling operations.

10. A parallel computing method, characterized in that: The method is implemented based on the system according to any one of claims 1 to 9, including: A parallel optical convolution module (2) is used to perform parallel convolution operations on N groups of multi-wavelength signals to be convolved output by the received multi-wavelength laser generation modulation module (1) in an optical operation domain, and forward propagation of a multi-layer neural network is realized by increasing the parallel flow of data, thereby obtaining a light intensity result for each wavelength; The photoelectric conversion module (3) is used to output the received light intensity result of each wavelength in the form of a voltage signal; Using the electrical processing control module (4) to post-process the received voltage signal to obtain a numerical result of the convolution operation, and controlling the multi-wavelength laser generation modulation module according to the numerical result and required data obtained from the outside to update the input data to be convolved; The data storage module (5) is used to store the operation results of each layer of the forward propagation and to transfer the convolution data required for the current parallel convolution operation.

Images