A photonic neural network system based on a multi-task neural network

Abstract

The application provides a photonic neural network system based on a multi-task neural network, which comprises: an optical computing module, taking a photonic artificial intelligence chip as a core, modulating an optical signal, and completing multiplication, summation and nonlinear operation of high-precision and high-speed optical analog quantities; a multi-task neural network module, having a main branch neural network and multiple secondary branch neural networks, and capable of simultaneously processing multiple classification tasks and regression tasks; and an optical computing communication and control module based on FPGA, connected with the optical computing module and the multi-task neural network module, and used for realizing real-time and high-speed communication of the optical computing module and the multi-task neural network module. The system has the characteristics of high bandwidth and low energy consumption, utilizes high-dimension parallel computing to greatly reduce the time of original neural network serial operation, has the advantages of large network scale and fast parallel operation speed, and can improve recognition accuracy and shorten training time on the basis of a single task.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a photonic neural network system based on a multi-task neural network. Background Technology

[0002] Currently, the computing power required for neural network algorithm models mainly relies on GPU servers and electronic neural network chips. However, electronic chips are limited by Moore's Law, with an update and iteration cycle of 12-18 months, which cannot keep up with the growth rate of computing power demand. Moreover, electronic chips suffer from the von Neumann bottleneck, meaning that the hardware framework of electronic chips causes neural networks to repeatedly read and move stored data during operation, increasing additional energy consumption and time costs.

[0003] In the field of optics, building neural networks based on analog frameworks can successfully avoid the von Neumann bottleneck and has the high bandwidth advantage that electronic signals do not have. It can make full use of the parallel processing capabilities of light to solve the problems of electronic neural networks.

[0004] With the development of technology, deep learning has become increasingly popular. Compared with traditional manual feature classification methods, deep learning methods improve classification accuracy and save manpower and resources. Multi-task deep learning methods have higher recognition accuracy and shorter training time than traditional single-task methods, thus improving efficiency. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a photonic neural network system based on a multi-task neural network. This system mitigates the limitations imposed by electronic chips on the computational speed of existing neural networks, and utilizes photonic artificial intelligence chips to achieve high-speed, parallel, low-power, and miniaturized neural networks.

[0006] This invention provides a photonic neural network system based on a multi-task neural network, comprising: an optical computing module, with a photonic artificial intelligence chip as its core, used for loading information of optical signals, modulating optical signals, and performing high-precision, high-speed multiplication, summation, and nonlinear operations of optical analog quantities;

[0007] The multi-task neural network module has a main branch neural network and multiple secondary branch neural networks added based on the main branch neural network according to the task. It is used to process digital electrical signals preprocessed by the FPGA-based optical computing communication and control module. It can process multiple classification tasks and regression tasks simultaneously, and extract the weight information of the network and input it into the FPGA-based optical computing communication and control module.

[0008] An FPGA-based optical computing communication and control module is connected to the optical computing module and the multi-task neural network module to enable real-time high-speed communication between the optical computing module and the multi-task neural network module, and to enable real-time updates of the weights of the optical computing module.

[0009] This invention uses photonic artificial intelligence chips to form an optical computing module, which, compared with existing electronic chips, expands bandwidth, reduces energy consumption, avoids heat generation problems, and breaks through the limitations of Moore's Law.

[0010] This invention utilizes multidimensional multiplexing technology in the design of photonic neural networks to enable parallel spatial transmission of different light pulses transmitted in a single time dimension by leveraging the spatial dimension. This significantly reduces the time required for serial computation in traditional neural networks and offers advantages such as large network scale and fast parallel computation speed.

[0011] This invention fully leverages the advantages of multi-task deep learning, enabling feature sharing among multiple tasks, thereby improving recognition accuracy on the basis of single-task training. Furthermore, simultaneous training of multiple tasks effectively shortens training time and improves efficiency. Attached Figure Description

[0012] Figure 1 System block diagram of the present invention;

[0013] Figure 2 System block diagram of the optical computing module;

[0014] Figure 3 Structure diagram of a multi-task neural network;

[0015] Figure 4 System diagram of FPGA-based optical computing, communication and control module; Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Unless otherwise specifically stated, the relative arrangement of the components and steps described in these embodiments does not limit the scope of the present invention. To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0017] The invention will now be further described with reference to the accompanying drawings.

[0018] Reference Figure 1The present invention provides a photonic neural network system based on a multi-task neural network, comprising: an optical computing module, a multi-task neural network module, and an FPGA-based optical computing communication and control module.

[0019] Reference Figure 2 The optical computing module of this invention, with a photonic artificial intelligence chip as its core, is used for loading optical signal information, modulating optical signals, and performing high-precision, high-speed multiplication, summation, and nonlinear operations of optical analog quantities. It includes a laser emitter, a photonic artificial intelligence chip, and a photoelectric converter.

[0020] Specifically, the laser emitter is connected to an FPGA-based optical computing communication and control module. Based on information received from the FPGA-based optical computing communication and control module, the laser emitter is controlled to emit light pulses. An optical fiber is used to connect to the photonic AI chip, providing it with continuous optical input. This invention's photonic AI chip employs multidimensional multiplexing technology, utilizing spatial dimensions to achieve parallel transmission in the design of the photonic neural network.

[0021] The photonic AI chip is composed of a cascaded array of Mach-Zehnder interferometers, which modulates the optical signal input from the optical transmitter in the following steps:

[0022] Step 11: Modulate the optical pulse according to the input of the multi-task neural network received from the FPGA-based optical computing communication and control module. The input of the multi-task neural network will be modulated onto the optical pulse.

[0023] Step 12: Modulate the optical pulse according to the weight information of the multi-task neural network received from the FPGA-based optical computing communication and control module. The weight information of the multi-task neural network will be modulated onto the optical pulse.

[0024] Step 13: In the optical domain, perform multiplication operations on the input and weights of the multi-task neural network.

[0025] Step 14: Perform a nonlinear operation on the result of the multiplication operation in step 13 in the optical domain.

[0026] Step 15: The result of step 14 is input into the photoelectric converter, which converts the optical signal containing the calculation result into an electrical signal, and then transmits the electrical signal to the FPGA-based optical computing communication and control module.

[0027] Reference Figure 3 The multi-task neural network module of this invention has a main branch neural network and multiple secondary branch neural networks added based on the main branch neural network according to the task. It can process multiple classification tasks and regression tasks simultaneously, and extract the weight information of the network and pass it into the FPGA-based optical computing communication and control module.

[0028] The main neural network is used to identify and extract features for each task.

[0029] The secondary neural network of the multi-task neural network module provides corresponding loss functions and optimization functions for different tasks, and adjusts the preset multi-task neural network model to complete the corresponding task functions.

[0030] During training, the training weights of the main branch neural network can be preset, and the training weights of the first main branch neural network are less than the training weights of the secondary branch neural networks.

[0031] When a new task stream arrives, depending on the number of new tasks, one or more new secondary neural networks with unique identification features are added. These new secondary neural networks are used for learning new tasks.

[0032] Compared with other processors, FPGAs have advantages such as high computational parallelism, flexible design, and low power consumption. They can also be used to optimize neural networks. Therefore, optical computing communication and control modules designed based on FPGA devices can make full use of the advantages of FPGAs to achieve real-time high-speed communication between optical computing modules and multi-task neural network modules.

[0033] The structure of the FPGA-based optical computing communication and control module can be as follows: Figure 4 As shown, the FPGA-based optical computing communication and control module is connected to the optical computing module and the multi-task neural network module to realize real-time high-speed communication between the optical computing module and the multi-task neural network module, and to realize real-time weight updates of the optical computing module. It includes: FPGA chip, AD / DA module, input / output interface module, clock module, and memory module.

[0034] The FPGA chip is used to process the input signal according to a preset parallelism to complete real-time high-speed communication between the optical computing module and the neural network.

[0035] The AD / DA module is connected to the FPGA chip, clock module and input / output interface module, and provides four-channel AD / DA conversion for digital-to-analog conversion between the optical computing module and the FPGA;

[0036] The input / output interface module is connected to the FPGA chip and the AD / DA module, and is used to expand the interface of the FPGA chip, providing a high-speed PCIe 3.0 interface, a network port and an SFP fiber optic interface;

[0037] The clock module is connected to the FPGA chip module and the AD / DA module, and is used to provide the operating clock for the FPGA chip;

[0038] The memory module is connected to the FPGA chip and is used to exchange stored data with the FPGA.

[0039] This invention is not limited to the embodiments described above. The above description of specific embodiments is intended to illustrate and explain the technical solutions of this invention. The specific embodiments described above are merely illustrative and not restrictive. Without departing from the spirit and scope of the claims, those skilled in the art can make many specific modifications based on the teachings of this invention, and these modifications all fall within the scope of protection of this invention.

Claims

1. A photonic neural network system based on a multi-task neural network, characterized in that... include: The optical computing module, with a photonic artificial intelligence chip at its core, is used for loading information into optical signals, modulating optical signals, and performing high-precision, high-speed multiplication, summation, and nonlinear operations on optical analog quantities. The multi-task neural network module has a main branch neural network and multiple secondary branch neural networks added based on the main branch neural network according to the task. It can process multiple classification and regression tasks simultaneously, and extract the weight information of the network and input it into the FPGA-based optical computing communication and control module. An FPGA-based optical computing communication and control module is connected to the optical computing module and the multi-task neural network module to realize real-time high-speed communication between the optical computing module and the multi-task neural network module, and to realize real-time updating of the weights of the optical computing module. The optical computing module includes: a laser, a photonic artificial intelligence chip, and a photoelectric converter. The laser emitter is connected to an FPGA-based optical computing communication and control module. Based on the information received from the FPGA-based optical computing communication and control module, the laser emitter is controlled to emit light pulses. The laser emitter is connected to the photonic artificial intelligence chip via an optical fiber to provide continuous light input to the photonic artificial intelligence chip. The photonic artificial intelligence chip is used to modulate the light signal input from the optical emitter. The secondary neural network of the multi-task neural network module provides corresponding loss functions and optimization functions for different tasks, and adjusts the preset multi-task neural network model to complete the corresponding task functions.

2. The photonic neural network system according to claim 1, characterized in that, The photonic artificial intelligence chip is composed of a cascaded array of Mach-Zehnder interferometers.

3. The photonic neural network system according to claim 2, characterized in that, The photonic AI chip is used to perform multiplication, summation, and nonlinear operations in the optical domain.

4. The photonic neural network system according to claim 2, characterized in that, The photonic AI chip employs multidimensional multiplexing technology, utilizing spatial dimensions to achieve parallel transmission in the design of the photonic neural network.

5. The photonic neural network system according to claim 1, characterized in that, The main branch neural network of the multi-task neural network module is used to identify and extract features for each task.

6. The photonic neural network system according to claim 1, characterized in that, In the multi-task neural network module, during training, the training weights of the main branch neural network can be randomly initialized or pre-set, and the training weights of the main branch neural network are less than the training weights of the secondary branch neural networks.

7. The photonic neural network system according to claim 1, characterized in that, The FPGA-based optical computing communication and control module includes: an FPGA chip, an AD / DA module, an input / output interface module, a clock module, and a memory module; wherein: The FPGA chip is used to process the input signal according to a preset parallelism to complete real-time high-speed communication between the optical computing module and the neural network. The AD / DA module is connected to the FPGA chip, clock module, and input / output interface module, providing a four-channel AD / DA conversion for digital-to-analog conversion between the optical computing module and the FPGA; The input / output interface module is connected to the FPGA chip and the AD / DA module, and is used to expand the interface of the FPGA chip, providing a high-speed PCIe 3.0 interface, a network port and an SFP fiber optic interface; The clock module is connected to the FPGA chip module and the AD / DA module, and is used to provide the operating clock for the FPGA chip; The memory module is connected to the FPGA chip and is used to exchange stored data with the FPGA.

8. The photonic neural network system according to any one of claims 1 to 7, characterized in that, The photonic neural network system is an optoelectronic hybrid integrated board, with the optical computing module forming the optical part and the multi-task neural network module and the FPGA-based optical computing communication and control module forming the electrical part.

Images