Method for obtaining optical computing convolutional neural network and electronic device

Abstract

The disclosure provides an acquisition method of an optical computing convolutional neural network and an electronic device, and relates to the technical field of computers. According to the convolution kernel dimension of each convolution layer in the convolutional neural network, a set of Mach-Zehnder interferometer convolution kernel calculation structures is acquired. According to the output layer dimension of the convolutional neural network, an output calculation structure of the Mach-Zehnder interferometer is acquired. According to the convolution layer number of the convolutional neural network and the set of Mach-Zehnder interferometer convolution kernel calculation structures, a convolution structure of the optical computing convolutional neural network is acquired. According to the convolution structure of the optical computing convolutional neural network and the output calculation structure of the Mach-Zehnder interferometer, a network structure of the optical computing convolutional neural network is acquired. Related parameters of the optical computing convolutional neural network are encoded into the network structure of the optical computing convolutional neural network, and the optical computing convolutional neural network is acquired. The convenience of constructing the optical computing convolutional neural network can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method and electronic device for acquiring optical computational convolutional neural networks. Background Technology

[0002] With the rapid development of high-performance computing hardware, various artificial intelligence algorithms have been widely applied and popularized in various industries. Among them, convolutional neural networks can be applied to image processing and related fields. Due to the increasing demand for computing power, new types of computing have also emerged. Among them, optical computing, as a new computing mode, is developing rapidly and has a huge number of application scenarios. Summary of the Invention

[0003] This disclosure provides a method and electronic device for acquiring optical computational convolutional neural networks. Its main purpose is to address the problem of poor convenience in large-scale integration of optical computational convolutional neural networks.

[0004] According to a first aspect of this disclosure, a method for obtaining an optical computational convolutional neural network is provided, comprising:

[0005] Based on the kernel dimension of each convolutional layer in the convolutional neural network, obtain the set of Mach-Zehnder interferometer convolutional kernel computational structures;

[0006] Based on the output layer dimension of the convolutional neural network, the output calculation structure of the Mach-Zehnder interferometer is obtained;

[0007] Based on the number of convolutional layers in the convolutional neural network and the set of convolutional kernel calculation structures of the Mach-Zehnder interferometer, the convolutional structure of the optical computing convolutional neural network is obtained.

[0008] Based on the convolutional structure of the optical computational convolutional neural network and the output computational structure of the Mach-Zehnder interferometer, the network structure of the optical computational convolutional neural network is obtained;

[0009] The relevant parameters of the optical computing convolutional neural network are encoded into the network structure of the optical computing convolutional neural network to obtain the optical computing convolutional neural network.

[0010] According to a second aspect of this disclosure, an apparatus for acquiring an optical computational convolutional neural network is provided, comprising:

[0011] The set acquisition unit is used to acquire the set of Mach-Zehnder interferometer convolution kernel calculation structures based on the convolution kernel dimension of each convolutional layer in the convolutional neural network.

[0012] The structure acquisition unit is used to acquire the output calculation structure of the Mach-Zehnder interferometer based on the output layer dimension of the convolutional neural network.

[0013] The structure acquisition unit is also used to acquire the convolution structure of the optical computing convolutional neural network based on the number of convolutional layers of the convolutional neural network and the set of Mach-Zehnder interferometer convolutional kernel calculation structures.

[0014] The structure acquisition unit is further configured to acquire the network structure of the optical computing convolutional neural network based on the convolutional structure of the optical computing convolutional neural network and the output computing structure of the Mach-Zehnder interferometer.

[0015] The network acquisition unit is used to encode the relevant parameters of the optical computing convolutional neural network into the network structure of the optical computing convolutional neural network, and acquire the optical computing convolutional neural network.

[0016] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect above.

[0020] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method described in the first aspect above.

[0021] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0022] Through this disclosure, the following methods are employed: A set of Mach-Zehnder interferometer convolutional kernel computational structures is obtained based on the convolutional kernel dimensions of each convolutional layer in the convolutional neural network; the output computational structure of the Mach-Zehnder interferometer is obtained based on the output layer dimensions in the convolutional neural network; the convolutional structure of an optical computational convolutional neural network is obtained based on the number of convolutional layers in the convolutional neural network and the set of Mach-Zehnder interferometer convolutional kernel computational structures; the network structure of the optical computational convolutional neural network is obtained based on the convolutional structure of the optical computational convolutional neural network and the output computational structure of the Mach-Zehnder interferometer; and the relevant parameters of the optical computational convolutional neural network are encoded into its network structure to obtain the optical computational convolutional neural network. Therefore, by designing the MZI convolutional kernel computation structure and the MZI output computation structure through the relevant parameters of the convolutional neural network, an optical computational convolutional neural network can be obtained. This provides a way to construct optical computational convolutional neural networks, improves the construction efficiency of optical computational convolutional neural networks, enables large-scale integration of optical computational convolutional neural networks, improves the convenience of constructing optical computational convolutional neural networks, and allows optical computational convolutional neural networks to be executed on optical computing chips, thereby increasing the application scope and processing efficiency of optical computational convolutional neural networks.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0024] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0025] Figure 1 A flowchart illustrating a method for obtaining an optical computational convolutional neural network according to an embodiment of this disclosure;

[0026] Figure 2 This is a schematic diagram illustrating an example structure of an MZI provided in an embodiment of the present disclosure;

[0027] Figure 3 A schematic diagram illustrating an example of another method for obtaining an optical computational convolutional neural network provided in this embodiment of the disclosure;

[0028] Figure 4 This is a schematic diagram illustrating an example of a convolution kernel performing convolution calculations, provided in an embodiment of this disclosure.

[0029] Figure 5 A schematic diagram illustrating an example of a first fully connected output layer implementation provided in an embodiment of this disclosure;

[0030] Figure 6This is a schematic diagram illustrating an example of a second fully connected output layer implementation provided in an embodiment of this disclosure;

[0031] Figure 7 This is a schematic diagram illustrating an example of convolution kernel computation provided in an embodiment of this disclosure;

[0032] Figure 8 This is a schematic diagram illustrating an example of a two-layer convolutional layer implementation provided in an embodiment of this disclosure;

[0033] Figure 9 This is a schematic diagram illustrating the structure of an optical computing convolutional neural network provided in an embodiment of this disclosure;

[0034] Figure 10 This is a schematic diagram of the structure of an optical computing convolutional neural network acquisition device provided in an embodiment of this disclosure. Detailed Implementation

[0035] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0036] The following describes a method and electronic device for acquiring an optical computational convolutional neural network according to embodiments of the present disclosure, with reference to the accompanying drawings.

[0037] Figure 1 This is a flowchart illustrating a method for obtaining an optical computational convolutional neural network according to an embodiment of this disclosure. Figure 1 As shown, the method includes the following steps:

[0038] Step 101: Obtain the set of Mach-Zehnder interferometer convolution kernel calculation structures based on the convolution kernel dimensions of each convolutional layer in the convolutional neural network;

[0039] According to some embodiments, the execution subject of this disclosure may be, for example, an electronic device. The name of the electronic device is not limited. The electronic device does not specifically refer to a particular fixed device. For example, when the structure of the electronic device changes, the electronic device may also change accordingly. For example, when the device identifier of the electronic device changes, the electronic device may also change accordingly. Furthermore, the execution subject of this disclosure may also be a network device, which, for example, may also be referred to as a server or a server cluster.

[0040] In some embodiments, a Convolutional Neural Network (CNN) can be a deep learning architecture primarily used to process data with a grid structure, such as images. CNNs have achieved great success in computer vision and are also widely used in natural language processing, speech recognition, and other fields. This CNN does not refer to a specific fixed neural network. For example, when the structure of a CNN changes, the CNN can also change accordingly. CNNs can, for example, automatically extract hierarchical features of data (from edge textures to high-level semantics) through local receptive fields and weight sharing mechanisms, significantly reducing the number of parameters and computational complexity, while leveraging pooling operations to enhance translation invariance and noise resistance.

[0041] According to some embodiments, a convolutional layer can be, for example, a core component of a CNN, which slides across the input data and performs convolution operations using convolutional kernels (also called filters) to extract local features. Different convolutional neural networks can correspond to different numbers of convolutional layers, and different convolutional layers can correspond to different convolutional kernels.

[0042] In some embodiments, a convolutional kernel can be, for example, a small matrix in a convolutional neural network used to extract local features from the input data (typically an image). This convolutional kernel is not specifically defined as a fixed kernel. For example, the kernel may change as its size changes. A convolutional layer may include at least one convolutional kernel.

[0043] According to some embodiments, the kernel dimension can be used, for example, to determine the specific behavior of convolution operations and the shape of the output feature map in a convolutional neural network. Different convolution kernels can correspond to different kernel dimensions. The kernel dimension in the embodiments of this disclosure does not specifically refer to a fixed dimension.

[0044] In some embodiments, the set of Mach-Zehnder interferometer convolution kernel computational structures can be, for example, a collection of at least one Mach-Zehnder interferometer convolution kernel computational structure. This Mach-Zehnder convolution kernel computational structure can, for example, be a computational structure using an MZI convolution kernel. This set of Mach-Zehnder convolution kernel computational structures is the same as the set of MZI convolution kernel computational structures, and this set of MZI convolution kernel computational structures does not specifically refer to a fixed set. For example, when the number of MZI convolution kernel computational structures changes, the set of MZI convolution kernel computational structures can also change accordingly. For example, when a particular MZI convolution kernel computational structure in the set changes, the set of MZI convolution kernel computational structures can also change accordingly.

[0045] According to some embodiments, a Mach-Zehnder interferometer (MZI) is an optical interferometer whose core idea is to split a beam of light into two beams, allow them to propagate along different paths, and then recombine them. The phase difference between the two beams is measured by observing the interference pattern formed after the recombining. An MZI can, for example, consist of two beam splitters and two phase shifters. Adjustable parameters are located in the phase shifters; changing the phase of the input light is achieved by altering voltage, temperature, etc., which are the parameters that need to be set in optical computational convolutional neural networks. A single MZI performs a matrix multiplication of an input 2D vector, outputting a 2D vector, such as... Figure 2 As shown, where E in For input data, E out For output data, an MZI mesh structure (composed of multiple cascaded MZIs) can perform n×n matrix multiplication on an n-dimensional input vector. The Mach-Zehnder interferometer convolution kernel computation structure can, for example, be the structure used in optical computational convolutional neural networks to complete the convolution computation process.

[0046] In some embodiments, the set of Mach-Zehnder interferometer convolution kernel computation structures can be obtained based on the convolution kernel dimensions of each convolutional layer in the convolutional neural network.

[0047] Step 102: Obtain the output calculation structure of the Mach-Zehnder interferometer based on the output layer dimension of the convolutional neural network;

[0048] According to some embodiments, the output layer in a convolutional neural network can be, for example, a fully connected layer, and the dimension of this output layer can depend on the structure of the convolutional neural network and the dimension of the input data. The output layer dimension of the convolutional neural network is not specifically a fixed dimension. For example, when the convolutional neural network changes, the output layer dimension can also change accordingly.

[0049] In some embodiments, the output computation structure of the Mach-Zehnder interferometer can be used, for example, to process convolution operations corresponding to multiple sets of convolution weights. The output computation structure of the Mach-Zehnder interferometer is not specifically a fixed structure; for example, when the output layer dimension of the convolutional neural network changes, the output computation structure of the Mach-Zehnder interferometer can also change accordingly.

[0050] Step 103: Obtain the convolutional structure of the optical computing convolutional neural network based on the number of convolutional layers in the convolutional neural network and the set of Mach-Zehnder interferometer convolutional kernel calculation structures.

[0051] In some embodiments, the number of convolutional layers can be used to indicate the total number of convolutional layers in a convolutional neural network, and this number does not specifically refer to a fixed number of layers. For example, the number of convolutional layers can also change accordingly when the structure of the convolutional neural network changes.

[0052] In some embodiments, the convolutional structure of an optical computing convolutional neural network can refer to, for example, the convolutional structure of the constructed optical computing convolutional neural network. This convolutional structure is not specifically defined by a fixed structure. For example, when the number of convolutional layers changes or the set of MZI convolutional kernel computational structures changes, the convolutional structure of the optical computing convolutional neural network can also change accordingly. Optical computing, as a novel computing model, offers more efficient computing power and development potential than classical computing in handling certain computational processes. Specifically, silicon-based optoelectronics provides an attractive platform for optical computing due to its high-speed signal processing capabilities and compatibility with existing microelectronic manufacturing processes. Integrating optical components on silicon chips allows for the creation of complex optoelectronic circuits that can perform neural network operations at the speed of light.

[0053] Step 104: Obtain the network structure of the optical computational convolutional neural network based on the convolutional structure of the optical computational convolutional neural network and the output computational structure of the Mach-Zehnder interferometer.

[0054] According to some embodiments, the network structure of an optical computational convolutional neural network can refer to, for example, the constructed network structure of an optical computational convolutional neural network. This network structure can be, for example, a combination of a convolutional structure and the output computation structure of an MZI (Multi-Level Ionizer). The network structure of this optical computational convolutional neural network does not specifically refer to a fixed structure. For example, when the convolutional structure or the output computation structure of the MZI changes, the network structure of the optical computational convolutional neural network can also change accordingly.

[0055] Step 105: Encode the relevant parameters of the optical computing convolutional neural network into the network structure of the optical computing convolutional neural network to obtain the optical computing convolutional neural network.

[0056] According to some embodiments, the relevant parameters may include, for example, parameters corresponding to the application scenario of the optical computing convolutional neural network, parameters of the convolutional neural network, and parameters of the MZI device. These relevant parameters do not specifically refer to any fixed parameter. For example, when the parameter value or parameter type changes, the relevant parameter may also change accordingly.

[0057] In some embodiments, the Optical Convolutional Neural Network (O-CNN) can be, for example, a computational architecture that combines optical computing and convolutional neural networks (CNNs), leveraging the high-speed propagation and parallel processing characteristics of light to improve computational efficiency. This optical computational convolutional neural network can, for example, be the network constructed in the embodiments of this disclosure.

[0058] This disclosure provides a method for constructing optical computational convolutional neural networks (OCNs). By determining the kernel dimensions of each convolutional layer in an OCN, a set of Mach-Zehnder interferometer (MZI) convolutional kernel computation structures is obtained. Similarly, by determining the output layer dimensions of an OCN, the output computation structure of an OZI is obtained. Furthermore, by encoding the relevant parameters of the OZI into its network structure, an OZI convolutional kernel computation structure and an MZI output computation structure can be designed using the parameters of the OCN. This method improves the construction efficiency of OZI, enables large-scale integration of OZI, and allows its execution on optical computing chips, thereby expanding its application scope and processing efficiency.

[0059] Furthermore, in one possible implementation of this embodiment, Figure 3 This is a flowchart illustrating another method for obtaining an optical computational convolutional neural network provided in an embodiment of this disclosure. Figure 3 As shown, the method includes the following steps:

[0060] Step 201: Obtain the set of Mach-Zehnder interferometer convolution kernel calculation structures based on the convolution kernel dimensions of each convolutional layer in the convolutional neural network;

[0061] The relevant processes can be described as above, and will not be repeated here.

[0062] According to some embodiments, the Mach-Zehnder interferometer convolution kernel calculation structure is obtained based on the convolution kernel dimension of each convolutional layer in the convolutional neural network, including:

[0063] Obtain the square dimension of the convolution kernel dimension of each convolutional layer in a convolutional neural network;

[0064] Using a quadratic-dimensional Mach-Zehnder interferometer mesh structure as the computational structure for each layer of the Mach-Zehnder interferometer convolution kernel, the MZI convolution kernel computational structure can be determined based on the kernel dimension. Improving the matching between the MZI convolution kernel computational structure and the kernel dimension can enhance the accuracy of the MZI convolution kernel computational structure determination.

[0065] In some embodiments, such as for image recognition scenarios, each convolution calculation process in a convolutional neural network is a process in which the convolution kernel performs element-wise multiplication and summation on a block of image within its field of view, and calculates and outputs one element value for a region containing n elements. When using n... 2 When using a 3D input MZI mesh structure, the output of the first waveguide position can be taken as the convolution output. By inputting the input information into the MZI mesh via light, the output of the first waveguide position can be measured, thus completing a convolution calculation at the speed of light. Therefore, optical computation of a convolutional neural network requires only one set of operational data to complete. Figure 4 This diagram illustrates how a single convolutional kernel performs a 2×2 convolution calculation. x1, x2, x3, and x4 are the input vectors, c1, c2, c3, and c4 are the convolutional kernels, and y1 is the output data of the convolutional kernel. The left side shows the logical process of the convolution calculation, while the right side shows the process using an MZI mesh structure. A 4×4 MZI mesh structure can be used, accommodating 4-dimensional vectors as input. The MZI convolutional kernel structure for each convolutional layer is set according to the dimension n of the kernel in each layer.

[0066] According to some embodiments, the number of convolutional layers in a convolutional neural network can be, for example, l, and the dimensions of each convolutional kernel can be obtained as n1, n2, ..., n. l It can be determined that the computational structure of the convolutional kernel in each layer is of dimension n. i 2 The MZI convolution kernel computation structure.

[0067] Step 202: Obtain the output calculation structure of the Mach-Zehnder interferometer based on the output layer dimension of the convolutional neural network;

[0068] The relevant processes can be described as above, and will not be repeated here.

[0069] According to some embodiments, the output computation structure of the Mach-Zehnder interferometer is obtained based on the output layer dimension of the convolutional neural network, including:

[0070] Obtain at least one output layer dimension in a convolutional neural network;

[0071] The maximum output layer dimension among at least one output layer dimensions is used as the dimension of the output computation structure of the Mach-Zehnder interferometer. Therefore, selecting the maximum output layer dimension among at least one output layer dimensions as the dimension of the output computation structure of the Mach-Zehnder interferometer can reduce the situation where the inaccuracy of the output computation structure dimension of the MZI makes it impossible to perform logical calculations using the MZI output computation structure, thereby improving the accuracy of optical computational convolutional neural network acquisition.

[0072] According to some embodiments, when implementing image classification tasks using convolutional neural networks, a final output layer is needed to calculate the output used for classification from the features extracted by the convolutional layers. For example, a fully connected layer can be used to map m features to j classification categories, thus achieving a matrix calculation with m inputs and j outputs. In optical computational convolutional neural networks, this can be accomplished using an MZI mesh structure. For example, an m-dimensional input MZI mesh structure can be used, taking the outputs of the first j waveguide positions as the fully connected output, allowing the computation of a fully connected layer to be completed at the speed of light. Similarly, compared to the two operands (multiplication and addition) of a classical computer, optical computation of a fully connected layer only requires one operand. Figure 5 The diagram shows a 2×4 fully connected output layer implemented using an MZI computational structure. The left side illustrates the logical computation process of the fully connected output layer, while the right side shows the implementation process of the MZI mesh structure. O represents the MZI output layer computation structure, and logit1 and logit2 are the outputs of the logical computation. For the output layer, the input dimension is smaller than the output dimension, as shown below. Figure 6 The diagram shows a 4×2 fully connected layer calculation. The MZI mesh structure can be configured with larger dimensions to complete the calculation.

[0073] According to some embodiments, the dimensions of the output layer of the obtained convolutional neural network may include k and p, and the larger of k and p can be determined as the dimension of the MZI output computation structure.

[0074] Step 203: Obtain the convolutional structure of the optical computing convolutional neural network based on the number of convolutional layers in the convolutional neural network and the set of Mach-Zehnder interferometer convolutional kernel calculation structures.

[0075] The relevant processes can be described as above, and will not be repeated here.

[0076] According to some embodiments, the convolutional structure of an optical computational convolutional neural network is obtained based on the number of convolutional layers in the convolutional neural network and the set of Mach-Zehnder interferometer convolutional kernel computational structures, including:

[0077] The subset of Mach-Zehnder interferometer convolution kernel computation structures belonging to the same convolution layer in the set of Mach-Zehnder interferometer convolution kernel computation structures is integrated to obtain the convolution layer corresponding to the number of convolution layers.

[0078] By performing ensemble operations on the convolutional layers corresponding to the number of convolutional layers, the convolutional structure of the optical computational convolutional neural network can be obtained. Therefore, ensemble operations on the convolutional structure and the output computation structure can improve the completeness and accuracy of the optical computational convolutional neural network acquisition.

[0079] According to some embodiments, when obtaining the convolutional layer corresponding to the number of convolutional layers, since MZI optical devices are easy to integrate and facilitate parallel high-speed computing, multiple convolutional kernels can be directly combined into a convolutional layer according to the required convolutional operations. The operation can be completed with only one input, achieving high-speed computing. For example... Figure 7 For a 4×4 input image, a 2×2 image is output after completing the convolution calculation process with a stride of 2 using a 2×2 convolution kernel. The left side shows the logical process of the convolution layer calculation, and the right side shows the optical computation implementation.

[0080] According to some embodiments, by integrating four convolutional kernels C1 together, the computation of the entire convolutional layer can be completed with a single input. The specific integration structure can be designed based on the input dimension and the convolutional kernel dimension. Here, C1 refers to... Figure 5 The MZI output calculation structure in the middle.

[0081] According to some embodiments, the method further includes:

[0082] The Mach-Zehnder interferometer kernel calculation structures within the subset of Mach-Zehnder interferometer kernel calculation structures for the same convolutional layer are arranged in parallel. Specifically, this can be achieved as follows: Figure 7 The multiple C1s shown.

[0083] In some embodiments, convolutional layers can be combined into a single convolutional operation. Specifically, a convolutional neural network may have multiple convolutional layers, and by integrating these multiple layers, optical computation of all convolutional operations can be achieved. Figure 8 As shown, for an 8×8 input image, a 2×2 image can be output after performing a convolution calculation with a stride of 2 using two layers of convolution kernels. For example, it can be shown as follows: Figure 8 As shown, the upper part represents the logical computation process of two convolutional layers, while the lower part represents the optical computation implementation. By integrating 16 C1 convolutional kernels and 4 C2 convolutional kernels, two convolutional layers can be completed with a single input. The specific integration structure can be designed based on the input dimension, the number of convolutional layers, and the convolutional kernel dimension.

[0084] Step 204: Obtain the network structure of the optical computational convolutional neural network based on the convolutional structure of the optical computational convolutional neural network and the output computational structure of the Mach-Zehnder interferometer.

[0085] The relevant processes can be described as above, and will not be repeated here.

[0086] According to some embodiments, an output computation structure can be added at the end of the convolutional structure of the optical computational convolutional neural network, which can complete the all-optical computational integration of the overall optical computational convolutional neural network's computational classification process. Figure 9 The diagram illustrates how, for an 8×8 input image, two 2×2 convolutional kernels are used to perform convolution calculations with a stride of 2, and an output layer is used to complete a 2-class classification task.

[0087] Step 205: Encode the relevant parameters of the optical computing convolutional neural network into the network structure of the optical computing convolutional neural network to obtain the optical computing convolutional neural network;

[0088] The relevant processes can be described as above, and will not be repeated here.

[0089] Step 206: Obtain visual data corresponding to the scene analysis requirements;

[0090] The relevant processes can be described as above, and will not be repeated here.

[0091] According to some embodiments, scene analysis requirement information can be used, for example, to indicate the operations performed using the optical computation convolutional neural network. This scene analysis requirement information does not specifically refer to any single fixed piece of information. It includes, but is not limited to, information such as image classification, object detection, and medical image segmentation, and can also be analysis requirement information in scenarios such as natural language processing and video analysis. Scene analysis requirement information may, for example, include image segmentation requirement information, image classification requirement information, etc.

[0092] In some embodiments, visual data may refer, for example, to image or video data captured by a visual sensor (such as a camera), which can be processed and analyzed by an optical computer convolutional neural network. This visual data does not specifically refer to any single, fixed set of data. For example, the visual data may change depending on whether the method of acquisition or the time of acquisition changes.

[0093] According to some embodiments, the relevant parameters of the optical computing convolutional neural network are encoded into the network structure of the optical computing convolutional neural network to obtain the optical computing convolutional neural network, including:

[0094] Encode the pixels of the input image onto the input light source;

[0095] The parameters of a convolutional neural network are encoded onto the Mach-Zehnder interferometer convolution kernel computation structure of an optical computational convolutional neural network, thus obtaining an optical computational convolutional neural network. Therefore, optical computational convolutional neural networks can be applied to image processing scenarios, expanding their application scope and improving their efficiency in image processing.

[0096] According to some embodiments, when performing image classification, the intensity of the input pixel can be set to the input light source, and the parameters of the MZI device can be set to the corresponding device.

[0097] Step 207: Input the visual data into the optical computing convolutional neural network for recognition and obtain the analysis results corresponding to the scene analysis requirements.

[0098] The relevant processes can be described as above, and will not be repeated here.

[0099] According to some embodiments, the method further includes:

[0100] The light source information of the target image is input into the optical computing convolutional neural network for calculation to obtain at least one output light intensity data.

[0101] Based on at least one output light intensity data, classification information corresponding to the target image is obtained. Therefore, optical computational convolutional neural networks can be used to obtain classification information. A fully optical implementation of the convolutional neural network can be achieved through a combination of MZI devices, enabling rapid implementation of convolutional neural network operations, completing classification tasks, and improving classification efficiency.

[0102] According to some embodiments, classification information corresponding to a target image is obtained based on at least one output light intensity data, including:

[0103] Obtain the weights corresponding to each output light intensity data in at least one output light intensity data set;

[0104] The classification information of the target image is obtained based on at least one output light intensity data and the weights corresponding to each output light intensity data. Therefore, the classification information can be determined based on the weights, which can improve the accuracy of the classification information acquisition.

[0105] Different output light intensity data can correspond to different weights, and the weights can be determined, for example, based on the intensity of the output light intensity data.

[0106] In some or related embodiments, visual data corresponding to scene analysis requirements can be acquired; the visual data is input into an optical computing convolutional neural network for recognition, and analysis results corresponding to scene analysis requirements are obtained. This enables all-optical convolutional neural network computation from image input to convolutional and output layers, which can improve image processing efficiency and complete image processing tasks quickly and efficiently.

[0107] According to embodiments of this disclosure, this disclosure also provides an apparatus for acquiring optical computational convolutional neural networks.

[0108] For example, Figure 10 This is a schematic diagram of the structure of an optical computing convolutional neural network acquisition device provided in an embodiment of the present disclosure. The optical computing convolutional neural network acquisition device 1000 includes: a set acquisition unit 1001, a structure acquisition unit 1002, and a network acquisition unit 1003; wherein,

[0109] The set acquisition unit 1001 is used to acquire the set of Mach-Zehnder interferometer convolution kernel calculation structures based on the convolution kernel dimension of each convolutional layer in the convolutional neural network.

[0110] The structure acquisition unit 1002 is used to acquire the output calculation structure of the Mach-Zehnder interferometer based on the output layer dimension of the convolutional neural network.

[0111] The structure acquisition unit 1002 is also used to acquire the convolution structure of the optical computing convolutional neural network based on the number of convolutional layers of the convolutional neural network and the set of structures calculated by the Mach-Zehnder interferometer convolutional kernels.

[0112] The structure acquisition unit 1002 is also used to acquire the network structure of the optical computing convolutional neural network based on the convolutional structure of the optical computing convolutional neural network and the output computing structure of the Mach-Zehnder interferometer.

[0113] The network acquisition unit 1003 is used to encode the relevant parameters of the optical computing convolutional neural network into the network structure of the optical computing convolutional neural network to acquire the optical computing convolutional neural network.

[0114] Furthermore, the structure acquisition unit 1002, when acquiring the Mach-Zehnder interferometer convolution kernel calculation structure based on the convolution kernel dimensions of each convolutional layer in the convolutional neural network, is specifically used for:

[0115] Obtain the square dimension of the convolution kernel dimension of each convolutional layer in a convolutional neural network;

[0116] By using a quadratic-dimensional Mach-Zehnder interferometer mesh structure as the calculation structure for each layer of the Mach-Zehnder interferometer convolution kernel, a set of Mach-Zehnder interferometer convolution kernel calculation structures is obtained.

[0117] Furthermore, the structure acquisition unit 1002, when acquiring the output calculation structure of the Mach-Zehnder interferometer based on the output layer dimension of the convolutional neural network, is specifically used for:

[0118] Obtain at least one output layer dimension in a convolutional neural network;

[0119] The largest output layer dimension among at least one output layer dimension is used as the dimension of the output computation structure of the Mach-Zehnder interferometer.

[0120] Furthermore, the structure acquisition unit 1002, when acquiring the convolutional structure of the optical computational convolutional neural network based on the number of convolutional layers of the convolutional neural network and the structure set calculated by the Mach-Zehnder interferometer convolutional kernel, is specifically used for:

[0121] The subset of Mach-Zehnder interferometer convolution kernel computation structures belonging to the same convolution layer in the set of Mach-Zehnder interferometer convolution kernel computation structures is integrated to obtain the convolution layer corresponding to the number of convolution layers.

[0122] An ensemble operation is performed on the convolutional layers corresponding to the number of convolutional layers to obtain the convolutional structure of the optical computing convolutional neural network.

[0123] Furthermore, the structure acquisition unit 1002 is also specifically used for:

[0124] The Mach-Zehnder interferometer kernel calculation structures in the subset of Mach-Zehnder interferometer kernel calculation structures for the same convolutional layer are set up in parallel.

[0125] Furthermore, the network acquisition unit 1003 is also specifically used for:

[0126] Acquire visual data corresponding to the scene analysis requirements;

[0127] Visual data is input into an optical computing convolutional neural network for recognition, and analysis results corresponding to the scene analysis requirements are obtained.

[0128] Furthermore, the network acquisition unit 1003 is used to encode the relevant parameters of the optical computing convolutional neural network into the network structure of the optical computing convolutional neural network. Specifically, when acquiring the optical computing convolutional neural network, it is used for:

[0129] Encode the pixels of the input image onto the input light source;

[0130] The parameters of the convolutional neural network are encoded onto the Mach-Zehnder interferometer convolution kernel computation structure of the optical computational convolutional neural network to obtain the optical computational convolutional neural network.

[0131] Furthermore, the network acquisition unit 1003 is also specifically used for:

[0132] The light source information of the target image is input into the optical computing convolutional neural network for calculation to obtain at least one output light intensity data.

[0133] Based on at least one output light intensity data, obtain the classification information corresponding to the target image.

[0134] Furthermore, when the network acquisition unit 1003 acquires classification information corresponding to the target image based on at least one output light intensity data, it is specifically used for:

[0135] Obtain the weights corresponding to each output light intensity data in at least one output light intensity data set;

[0136] Based on at least one output light intensity data and the weights corresponding to each output light intensity data, the classification information corresponding to the target image is obtained.

[0137] It should be noted that the description of the features in the embodiment corresponding to the optical computing convolutional neural network acquisition device can be found in the relevant description of the embodiment corresponding to the optical computing convolutional neural network acquisition method, and will not be repeated here.

[0138] Embodiments of this disclosure also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above embodiments of the method for acquiring an optical computational convolutional neural network.

[0139] Embodiments of this disclosure also provide a computer-readable storage medium storing a computer program configured to execute the steps in any of the above embodiments of the method for obtaining an optical computational convolutional neural network at runtime.

[0140] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0141] The embodiments of this disclosure also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the optical computational convolutional neural network acquisition method.

[0142] Embodiments of this disclosure also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the optical computational convolutional neural network acquisition method.

[0143] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0144] The above provides a detailed description of a method for obtaining an optical computational convolutional neural network. Specific examples have been used to illustrate the principles and implementation methods of this disclosure. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this disclosure without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this disclosure.

Claims

1. A method for obtaining an optical computational convolutional neural network, characterized in that, include: Based on the kernel dimension of each convolutional layer in the convolutional neural network, obtain the set of Mach-Zehnder interferometer convolutional kernel computational structures; Based on the output layer dimension of the convolutional neural network, the output calculation structure of the Mach-Zehnder interferometer is obtained; Based on the number of convolutional layers in the convolutional neural network and the set of convolutional kernel calculation structures of the Mach-Zehnder interferometer, the convolutional structure of the optical computing convolutional neural network is obtained. Based on the convolutional structure of the optical computational convolutional neural network and the output computational structure of the Mach-Zehnder interferometer, the network structure of the optical computational convolutional neural network is obtained; The relevant parameters of the optical computing convolutional neural network are encoded into the network structure of the optical computing convolutional neural network to obtain the optical computing convolutional neural network.

2. The method according to claim 1, characterized in that, The process of obtaining the Mach-Zehnder interferometer convolution kernel calculation structure set based on the convolution kernel dimensions of each convolutional layer in the convolutional neural network includes: Obtain the square dimension of the convolution kernel dimension of each convolutional layer in the convolutional neural network; The quadratic-dimensional Mach-Zehnder interferometer mesh structure is used as the Mach-Zehnder interferometer convolution kernel calculation structure for each convolutional layer to obtain the set of Mach-Zehnder interferometer convolution kernel calculation structures.

3. The method according to claim 1, characterized in that, The step of obtaining the output calculation structure of the Mach-Zehnder interferometer based on the output layer dimension of the convolutional neural network includes: Obtain at least one output layer dimension in the convolutional neural network; The largest output layer dimension among the at least one output layer dimensions is used as the dimension of the output calculation structure of the Mach-Zehnder interferometer.

4. The method according to claim 1, characterized in that, The step of obtaining the convolutional structure of the optical computational convolutional neural network based on the number of convolutional layers of the convolutional neural network and the set of Mach-Zehnder interferometer convolutional kernel calculation structures includes: The subset of Mach-Zehnder interferometer convolution kernel calculation structures belonging to the same convolution layer in the set of Mach-Zehnder interferometer convolution kernel calculation structures is integrated to obtain the convolution layer corresponding to the number of convolution layers. An ensemble operation is performed on the convolutional layers corresponding to the specified number of convolutional layers to obtain the convolutional structure of the optical computing convolutional neural network.

5. The method according to claim 4, characterized in that, The method further includes: The Mach-Zehnder interferometer kernel calculation structures in the subset of Mach-Zehnder interferometer kernel calculation structures for the same convolutional layer are arranged in parallel.

6. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Acquire visual data corresponding to the scene analysis requirements; The visual data is input into the optical computing convolutional neural network for recognition, and analysis results corresponding to the scene analysis requirements are obtained.

7. The method according to claim 6, characterized in that, The step of encoding the relevant parameters of the optical computing convolutional neural network into the network structure of the optical computing convolutional neural network to obtain the optical computing convolutional neural network includes: Encode the pixels of the input image onto the input light source; The parameters of the convolutional neural network are encoded onto the Mach-Zehnder interferometer convolution kernel computation structure of the optical computational convolutional neural network to obtain the optical computational convolutional neural network.

8. The method according to claim 7, characterized in that, The method further includes: The light source information of the target image is input into the optical computing convolutional neural network for calculation to obtain at least one output light intensity data; Based on the at least one output light intensity data, obtain the classification information corresponding to the target image.

9. The method according to claim 8, characterized in that, The step of obtaining the classification information corresponding to the target image based on the at least one output light intensity data includes: Obtain the weights corresponding to each output light intensity data in the at least one output light intensity data; Based on the at least one output light intensity data and the weights corresponding to each output light intensity data, the classification information corresponding to the target image is obtained.

10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method for obtaining an optical computational convolutional neural network as described in any one of claims 1 to 9.

Images