A Convolution Calculation Method and Device Based on a Photonic Computing Chip

Abstract

The present application discloses a convolution calculation method and device based on a photonic computing chip. The method is applicable to a convolution calculation device of a photonic computing chip, and the method includes: obtaining a weight matrix and loading the weight matrix into an interferometer array in the photonic computing chip; obtaining a target input vector to be processed; inputting the target input data into an input end, and using the target input data by the input end to perform a linear calculation of a vector-weight matrix to obtain an output vector; extracting elements belonging to the same row from each output vector and summing them to obtain a convolution calculation result between a convolution kernel and a target feature map. The present application uses a photonic computing chip to implement the linear calculation of a vector and a weight matrix, can simulate the operation between matrices in a convolutional neural network, and reduces the number of operations of the photonic computing chip and improves the calculation efficiency by optimizing the weight matrix extraction method.

Description

Technical Field

[0001] This application relates to the field of photon chip technology, and particularly to a convolution calculation method and device based on a photon computing chip. Background Art

[0002] A large number of matrix linear multiplication operations are included in the Artificial Neural Network (ANN) algorithm. To achieve massive data training and inference, AI chips such as GPUs, TPUs, FPGAs, ASICs, and brain-like chips have emerged. However, current AI chips still rely on electronic computing technology. With the failure of Moore's Law, it is difficult for electronic AI chips to meet the growing future computing power requirements. Therefore, researchers have proposed a non-Von Neumann architecture photon computing chip, which replaces electrical signals with optical signals to achieve high-speed, high-parallel, and low-power Optical Neural Network (ONN) operations.

[0003] In traditional convolution operations, taking a 3×3 convolution kernel as an example for illustration, as Figure 1 shown. After the feature matrices [a11, a12, a13], [a21, a2, a23], [a31, a32, a33] in the sliding window are convolved with the convolution kernel [w11, w12, w13], [w21, w22, w23], [w31, w32, w33], we get (a11w11 + a12w12 + a13w13) + (a21w21 + a22w22 + a23w23) + (a31w31 + a32w32 + a33w33). Using a photon computing chip to implement a 3×3 convolution kernel W. At this time, the U, Σ, and V* matrices are all 3×3, and each requires 3×(3 - 1) / 2 = 3 interferometers, for a total of 9 interferometers.

[0004] Since the input of the photon computing chip is a one-dimensional vector, the feature map in the sliding window needs to be input three times, namely [a11, a12, a13], [a21, a22, a23], and [a31, a32, a33] in sequence. The calculation results are as shown in formulas (1)-(3).

[0005] Formula (1):

[0006] Formula (2):

[0007] Formula (3):

[0008] Comparing the above results, it can be seen that only the first row of the output of formula (1) is valid, corresponding to the detector No. 1 of the photon computing chip. Only the second row of the output of formula (2) is valid, corresponding to the detector No. 2 of the photon computing chip. Only the first row of the output of formula (3) is valid, corresponding to the detector No. 3 of the photon computing chip. If there are n convolution kernels (usually n > m), under this sliding window, the photon computing chip has to process 3n times, resulting in too many computing times of the photon computing chip. Summary of the Invention

[0009] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides a convolution calculation method and device based on a photon computing chip.

[0010] According to one aspect of the embodiments of the present application, a convolution calculation method based on a photon computing chip is provided. The method is applicable to a convolution calculation device of a photon computing chip. The convolution calculation device includes an optical signal input end and an optical detector output end. The input end and the output end are connected by an interferometer array arranged in cascade. The method includes:

[0011] Obtain a weight matrix and load the weight matrix into the interferometer array in the photon computing chip;

[0012] Obtain a target input vector to be processed, where the target input vector is obtained from a target feature map;

[0013] Input the target input vector into the input end, and use the target input vector by the input end to perform a linear calculation of the vector-weight matrix to obtain an output vector;

[0014] Extract the elements belonging to the same row from each output vector and sum them to obtain the convolution calculation result between the convolution kernel and the target feature map.

[0015] Further, the obtaining of the weight matrix includes:

[0016] Extract the i-th row parameters from n m×m convolution kernels, and generate an m×n weight matrix corresponding to the i-th row parameters, where the number of weight matrices corresponding to the i-th row parameters is m;

[0017] Load the weight matrix corresponding to the i-th row parameters into the interferometer array of the photon computing chip, where m, n, and i are all integers greater than or equal to 1.

[0018] Further, the obtaining of the target input vector to be processed includes:

[0019] Obtain a target feature map to be processed;

[0020] Analyze the target feature map using a preset sliding window to obtain m feature row vectors to be processed, where each feature row vector includes m elements;

[0021] Extract the feature row vector of the i-th row, convert the feature row vector into a feature column vector corresponding to the i-th row, and determine the feature column vector as the target input vector.

[0022] Further, inputting the target input vector into the input end, and the input end uses the target input vector to perform a linear calculation of a vector-weight matrix to obtain an output vector, including:

[0023] Input the feature column vector corresponding to the i-th row into the input end, so that the input end performs a linear multiplication operation with the weight matrix corresponding to the i-th row to obtain the output vector corresponding to the i-th row;

[0024] Repeat running the photonic computing chip until m output vectors are obtained, where the output vector includes n elements.

[0025] Further, extracting the elements belonging to the same row from each output vector for summation to obtain the convolution calculation result between the convolution kernel and the target feature map, including:

[0026] Extract the elements belonging to the same row from the m output vectors for summation to obtain the output values corresponding to the m output vectors;

[0027] Determine the output values corresponding to the m output vectors as the calculation result.

[0028] According to another aspect of the embodiments of the present application, there is also provided a convolution calculation device based on a photonic computing chip, including: an optical signal input end and a photodetector output end, and the input end and the output end are connected by an interferometer array arranged in cascade, and a weight matrix is loaded in the interferometer array;

[0029] The input end is used to obtain a target input vector and perform a linear calculation of a vector-weight matrix using the target input vector to obtain an output vector;

[0030] The output end is used to extract the elements belonging to the same row from each output vector for summation to obtain the convolution calculation result between the convolution kernel and the target feature map.

[0031] Further, the weight matrix is generated according to a unitary matrix, a diagonal matrix, and a complex conjugate matrix corresponding to the unitary matrix, the dimension of the weight matrix is m×n, the dimension of the unitary matrix is m×m, the dimension of the diagonal matrix is m×n, and the dimension of the complex conjugate matrix is n×n.

[0032] Further, the number of interferometers in the interferometer array corresponding to the unitary matrix is calculated based on the following formula:

[0033] m1 = n(n - 1) / 2, where m1 is the number of interferometers in the interferometer array corresponding to the unitary matrix.

[0034] Further, the number of interferometers in the interferometer array corresponding to the diagonal matrix is calculated based on the following formula:

[0035] m2 = 3n(3n - 1) / 2, where m2 is the number of interferometers in the interferometer array corresponding to the diagonal matrix.

[0036] According to another aspect of the embodiments of the present application, there is also provided a storage medium, which includes a stored program that executes the above steps when the program runs.

[0037] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus; where: the memory is used to store a computer program; the processor is used to execute the steps in the above method by running the program stored on the memory.

[0038] The embodiments of the present application also provide a computer program product containing instructions, which when running on a computer, causes the computer to execute the steps in the above method.

[0039] The above technical solutions provided by the embodiments of the present application have the following advantages compared with the prior art: The present application uses a photonic computing chip to achieve linear calculation of vectors and weight matrices, can simulate the operation between matrices in a convolutional neural network, and by optimizing the weight matrix extraction method, reduces the number of operations of the photonic computing chip and improves the calculation efficiency. Description of the Drawings

[0040] The drawings here are incorporated into the specification and form a part of this specification, showing the embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0042] Figure 1 It is a schematic diagram of traditional convolution calculation provided by the embodiments of the present application;

[0043] Figure 2Schematic diagram of a convolution calculation device based on a photonic computing chip provided by an embodiment of the present application;

[0044] Figure 3 Flowchart of a convolution calculation method based on a photonic computing chip provided by an embodiment of the present application;

[0045] Figure 4 Schematic diagram of convolution calculation provided by an embodiment of the present application;

[0046] Figure 5 Schematic diagram of convolution calculation provided by an embodiment of the present application;

[0047] Figure 6 Schematic diagram of convolution calculation provided by an embodiment of the present application;

[0048] Figure 7 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present application.

[0050] It should be noted that in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another similar entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0051] Based on the above technical problems, the embodiments of the present application provide a convolution calculation method and device based on a photonic computing chip. The method provided by the embodiments of the present invention can be applied to any required electronic device. For example, it can be an electronic device such as a server, a terminal, etc., which is not specifically limited herein. For the convenience of description, it will be hereinafter simply referred to as an electronic device.

[0052] Figure 2 FIG. is a schematic diagram of a convolution calculation device based on a photonic computing chip provided by an embodiment of the present application. This device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. As Figure 2 shown, this device includes:

[0053] An optical signal input end and a photodetector output end, with the input end and the output end connected by an interferometer array arranged in cascade. A weight matrix is loaded in the interferometer array;

[0054] The input end is used to obtain a target input vector and perform a linear calculation of the vector-weight matrix using the target input vector to obtain an output vector;

[0055] The output end is used to extract the elements belonging to the same row from each output vector and sum them to obtain the convolution calculation result between the convolution kernel and the target feature map.

[0056] In the embodiments of the present application, the number of interferometers in the interferometer array is determined according to the weight matrix. The weight matrix is generated based on a unitary matrix, a diagonal matrix, and the complex conjugate matrix corresponding to the unitary matrix. The dimensions of the unitary matrix, the diagonal matrix, and the complex conjugate matrix are respectively determined according to the number of convolution kernels in the input layer.

[0057] In the embodiments of the present application, taking a 3×3 convolution kernel as an example, the dimension of the weight matrix is n×3, the dimension of the unitary matrix is n×n, the dimension of the diagonal matrix is n×3, and the dimension of the complex conjugate matrix is 3×3, where n is the number of convolution kernels.

[0058] It should be noted that in the embodiments of the present application, when designing the photonic computing chip, the number of interferometers in each corresponding interferometer array is determined based on the dimensions of the unitary matrix, the diagonal matrix, and the complex conjugate matrix. During the operation of the photonic computing chip, the set interferometer array can not only achieve conventional convolution calculations, but also reduce the number of calculations of the photonic computing chip.

[0059] In the embodiments of the present application, the number of interferometers in the interferometer array corresponding to the unitary matrix is calculated based on the following formula: m1 = n(n - 1) / 2, where m1 is the number of interferometers in the interferometer array corresponding to the unitary matrix.

[0060] In the embodiment of the present application, the number of interferometers in the interferometer array corresponding to the diagonal matrix is calculated based on the following formula: m2 = 3n(3n - 1) / 2, where m2 is the number of interferometers in the interferometer array corresponding to the diagonal matrix.

[0061] It should be noted that in order to reduce the computational complexity, the matrix dimension implemented by the photonic computing chip in the embodiment of the present application is n×3. Then, the U matrix is n×n and requires n(n - 1) / 2 interferometers. The Σ matrix is n×3 and requires 3n(3n - 1) / 2 interferometers. The V* matrix is 3×3 and requires 3 interferometers. n is the number of convolution kernels. The interferometers in the embodiment of the present application are Mach–Zehnder Interferometers (abbreviation: MZI).

[0062] According to an aspect of the embodiment of the present application, a method embodiment of a convolution calculation method based on a photonic computing chip is provided. Figure 3 The flowchart of a convolution calculation method based on a photonic computing chip provided for the embodiment of the present application is as Figure 3 shown, and the method includes:

[0063] Step S11, obtain a weight matrix and load the weight matrix into the interferometer array in the photonic computing chip.

[0064] The method provided by the embodiment of the present application is applied to a convolution calculation device of a photonic computing chip. The convolution calculation device includes an optical signal input end and a photodetector output end. The input end and the output end are connected by an interferometer array arranged in cascade. Among them, the photonic computing chip can determine the data sent by the requesting device as the target input data to be processed. Among them, the target input data to be processed can be text data, image data, etc.

[0065] It should be noted that since the matrix dimension of the weight matrix is m×n, it is necessary to insert weight values into the interferometer array in the optical neural network. Among them, the weight values can be weight values preset by developers. Specifically, the convolution kernel parameters corresponding to the input layer can be obtained, the corresponding weight values can be obtained based on the convolution kernel parameters, and the weight matrix can be constructed based on the weight values.

[0066] In the embodiment of the present application, obtaining the weight matrix includes the following steps A1 - A2:

[0067] Step A1, extract the i-th row parameters from n m×m convolution kernels and generate an m×n weight matrix corresponding to the i-th row parameters. Among them, the number of weight matrices corresponding to the i-th row parameters is m.

[0068] Step A2, load the weight matrix corresponding to the i-th row of parameters into the interferometer array of the photon computing chip, where m, n, and i are all integers greater than or equal to 1.

[0069] Step S12, obtain the target input vector to be processed, where the target input vector is obtained from the target feature map.

[0070] In the embodiment of the present application, Step S12, obtaining the target input vector to be processed includes the following steps B1 - B3:

[0071] Step B1, obtain the target feature map to be processed.

[0072] Step B2, analyze the target feature map using a preset sliding window to obtain m feature row vectors to be processed, where each feature row vector includes m elements.

[0073] Step B3, extract the feature row vector of the i-th row, convert the feature row vector into the corresponding feature column vector of the i-th row, and determine the feature column vector as the target input vector.

[0074] In the embodiment of the present application, the photon computing chip determines a preset sliding window during operation. The length of the preset sliding window is the same as the number of convolutional kernels in the output layer. For example, when there are 3 convolutional kernels in the output layer, the length of the preset sliding window is 3 and the width is 1. Use the preset sliding window to extract multiple feature row vectors from the target feature map. For example: the target feature map includes multiple feature vectors, and the length of the preset sliding time window is 3. Therefore, use the preset sliding window to traverse each row of feature vectors, extract 3 feature vectors (elements) in order from multiple feature vectors, and determine the 3 feature vectors (elements) as the first row of feature row vectors until all feature vectors are extracted to obtain m feature row vectors.

[0075] As an example, when there are n convolutional kernels, feature K can be extracted from the target feature map through the n convolutional kernels, perform convolutional calculation on feature K, and finally obtain m feature row vectors. Then, each feature row vector is converted to obtain m feature column vectors, and the feature column vectors are determined as the target input vectors.

[0076] Step S13, input the target input vector into the input end, and the input end uses the target input vector to perform linear calculation of the vector - weight matrix to obtain an output vector.

[0077] In the embodiment of the present application, Step S13, inputting the target input vector into the input end and the input end using the target input vector to perform linear calculation of the vector - weight matrix to obtain an output vector includes the following steps C1 - C2:

[0078] Step C1, input the feature column vector corresponding to the i-th row into the input end, so that the input end performs a linear multiplication operation with the weight matrix corresponding to the i-th row, and obtain the output vector corresponding to the i-th row.

[0079] Step C2, repeatedly run the photonic computing chip until m output vectors are obtained, where each output vector includes n elements.

[0080] In the embodiment of the present application, when the photonic computing chip is running, first load the weight values of the first row of n convolutional kernels into the interferometer array of the photonic computing chip. As Figure 4 shown, the input vector of the photonic computing chip is the first row vector [a11, a12, a13] within the preset sliding window, and the matrix operation result obtained is as shown in the formula.

[0081]

[0082] The result of the first row of the formula corresponds to the dot product accumulation of the target input data and the first row of the first convolutional kernel. The result of the n-th row corresponds to the dot product accumulation of the target input data and the first row of the n-th convolutional kernel. The results of the n rows are all valid and are saved in the memory for subsequent calls.

[0083] Secondly, as Figure 5 shown, load the data of the second row of n convolutional kernels into the interferometer array of the photonic computing chip. The input vector of the photonic computing chip is the second row [a21, a22, a23] within the preset sliding window, and calculate according to [a21, a22, a23] and the weight matrix to obtain the calculation result of the second row.

[0084] Finally, as Figure 6 shown, load the data of the third row of n convolutional kernels into the interferometer array of the photonic computing chip. The input vector of the photonic computing chip is the third row [a31, a32, a33] within the preset sliding window, and calculate according to [a31, a32, a33] and the weight matrix to obtain the calculation result of the third row.

[0085] Step S14, extract the elements belonging to the same row from each output vector and sum them to obtain the convolution calculation result between the convolutional kernel and the target feature map.

[0086] In the embodiment of the present application, Step S14, extract the elements belonging to the same row from each output vector and sum them to obtain the convolution calculation result between the convolutional kernel and the target feature map, including the following steps D1 - D2:

[0087] Step D1, extract the elements belonging to the same row from m output vectors and sum them to obtain the output values corresponding to the m output vectors.

[0088] Step D2: Determine the output values corresponding to the m output vectors as the convolution calculation result.

[0089] In the embodiment of the present application, after obtaining the output vector corresponding to each feature row vector, then perform cumulative summation on multiple elements belonging to the same row in the output vector of this row to obtain the output value. Finally, through the calculation of the output vector corresponding to each feature row vector, the convolution calculation result corresponding to the target feature map is obtained. Therefore, the present application uses a photonic computing chip to implement the linear calculation of vectors and weight matrices, can simulate the operation between matrices in a convolutional neural network, and by optimizing the weight matrix extraction method, reduces the number of operations of the photonic computing chip and improves the computing efficiency.

[0090] In the embodiment of the present application, after the interferometer array completes the calculation, the calculation result will be transmitted to the output layer, and the output layer will output the calculation result.

[0091] In the embodiment of the present application, when designing the photonic computing chip, determine the dimensions corresponding to the unitary matrix, diagonal matrix, and complex conjugate matrix according to the number of convolution kernels, and determine the number of interferometers in the corresponding interferometer array based on the dimensions corresponding to the unitary matrix, diagonal matrix, and complex conjugate matrix. During the operation of the photonic computing chip, the set interferometer array can not only implement conventional convolution calculations, but also reduce the number of calculations of the photonic computing chip. Compared with the prior art, the number of calculations of the photonic computing chip in the embodiment of the present application is 1 / n of the prior art, and at the same time, the computing efficiency of the photonic computing chip is improved.

[0092] In the embodiment of the present application, the method can also implement convolution kernel operations of other sizes. For example, for n m×m convolution kernels, it is necessary to run the photonic computing chip of n×m dimension m times, and the one-dimensional vector input to the photonic computing chip contains m elements. Specifically, when n is an integer multiple of m, the n×m dimensional photonic computing chip is decomposed into n / m m×m dimensional photonic computing chips. The input vector of each photonic computing chip is still m elements, but a beam splitter device is required to first divide the m input optical paths into 2m optical paths.

[0093] In the embodiment of the present application, it is also possible to first traverse the target input data graph and then refresh the convolution kernel. For example, when the photonic computing chip loads the weights of the first row of n convolution kernels, let step = 1, and the input vectors are [a11, a12, a13], [a12, a13, a14], [a13, a14, a15]... [aN(N - 2), aN(N - 1), aNN] in sequence. After the input signal traversal, update the photonic computing chip to the second row of n convolution kernels.

[0094] The embodiment of the present application also provides an electronic device, such as Figure 7As shown, the electronic device may include: a processor 1501, a communication interface 1502, a memory 1503, and a communication bus 1504. Among them, the processor 1501, the communication interface 1502, and the memory 1503 communicate with each other through the communication bus 1504.

[0095] The memory 1503 is used to store computer programs.

[0096] When the processor 1501 is used to execute the computer programs stored on the memory 1503, the steps of the above embodiments are implemented.

[0097] The communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0098] The communication interface is used for communication between the above terminal and other devices.

[0099] The memory may include a Random Access Memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0100] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0101] In another embodiment provided by the present application, a computer-readable storage medium is further provided. Instructions are stored in the computer-readable storage medium, and when it runs on a computer, it causes the computer to execute the convolution calculation method based on a photonic computing chip described in any of the above embodiments.

[0102] In another embodiment provided by the present application, a computer program product containing instructions is further provided. When it runs on a computer, it causes the computer to execute the following convolution calculation method based on a photonic computing chip. The method includes:

[0103] Obtain a weight matrix and load the weight matrix into an interferometer array in the photonic computing chip;

[0104] Obtain a target input vector to be processed, where the target input vector is obtained from a target feature map;

[0105] Input the target input vector into an input end, and the input end uses the target input vector to perform a linear calculation of the vector-weight matrix to obtain an output vector;

[0106] Extract the elements belonging to the same row from each output vector and sum them to obtain the convolution calculation result between the convolution kernel and the target feature map.

[0107] Further, obtaining the weight matrix includes:

[0108] Extract the parameters of the i-th row from n m×m convolution kernels, and generate an m×n weight matrix corresponding to the parameters of the i-th row. The number of weight matrices corresponding to the parameters of the i-th row is m;

[0109] Load the weight matrix corresponding to the parameters of the i-th row into the interferometer array of the photonic computing chip, where m, n, and i are all integers greater than or equal to 1.

[0110] Further, obtaining the target input vector to be processed includes:

[0111] Obtain a target feature map to be processed;

[0112] Use a preset sliding window to analyze the target feature map to obtain m feature row vectors to be processed, where each feature row vector includes m elements;

[0113] Extract the feature row vector of the i-th row, convert the feature row vector into a feature column vector corresponding to the i-th row, and determine the feature column vector as the target input vector.

[0114] Further, inputting the target input vector into the input end, and the input end uses the target input vector to perform a linear calculation of the vector-weight matrix to obtain an output vector, includes:

[0115] Input the feature column vector corresponding to the i-th row into the input end, so that the input end performs a linear multiplication operation with the weight matrix corresponding to the i-th row to obtain the output vector corresponding to the i-th row;

[0116] Repeat running the photonic computing chip until m output vectors are obtained, where each output vector includes n elements.

[0117] Furthermore, extract the elements belonging to the same row from each output vector and sum them to obtain the convolution calculation result between the convolution kernel and the target feature map, including:

[0118] Extract the elements belonging to the same row from the m output vectors and sum them to obtain the output values corresponding to the m output vectors;

[0119] Determine the output values corresponding to the m output vectors as the convolution calculation result.

[0120] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that the computer can access, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state disk SolidState Disk), etc.

[0121] The above are only the preferred embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

[0122] The above are only specific embodiments of the present application, enabling those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but rather will be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A convolutional computing method based on a photonic computing chip, characterized in that, The method is applicable to the convolution calculation device of a photonic computing chip. The convolution calculation device includes an optical signal input end and an optical detector output end. The input end and the output end are connected by an interferometer array arranged in cascade. The method includes: Obtain a weight matrix and load the weight matrix into the interferometer array in the photonic computing chip; Obtain a target input vector to be processed, where the target input vector is obtained from a target feature map; Input the target input vector into the input end, and use the target input vector through the input end to perform a linear calculation of the vector-weight matrix to obtain an output vector; Extract the elements belonging to the same row from each output vector and sum them to obtain the convolution calculation result between the convolution kernel and the target feature map; The obtaining of the target input vector to be processed includes: Obtain a target feature map to be processed; analyze the target feature map using a preset sliding window to obtain m to-be-processed feature row vectors, where each feature row vector includes m elements; extract the feature row vector of the i-th row, convert the feature row vector into the corresponding feature column vector of the i-th row, and determine the feature column vector as the target input vector; The inputting of the target input vector into the input end and using the target input vector through the input end to perform a linear calculation of the vector-weight matrix to obtain an output vector includes: Input the feature column vector corresponding to the i-th row into the input end, so that the input end performs a linear multiplication operation with the weight matrix corresponding to the i-th row to obtain the output vector corresponding to the i-th row; Repeat running the photonic computing chip until m output vectors are obtained, where each output vector includes n elements; Wherein, the calculation formula is as follows: The result of the first row in the formula corresponds to the dot product accumulation of the target input data and the first row of the first convolution kernel, and the result of the n-th row corresponds to the dot product accumulation of the target input data and the first row of the n-th convolution kernel; a 11 、a 12 、a 13 are the feature vectors located in the first row of the target feature map; w 11 、w 12 、w 13 is the weight matrix of the first row in the first convolutional kernel, w 21 、w 22 、w 23 is the weight matrix of the first row in the second convolutional kernel, and so on, w n1 、w n2 、w n3 is the weight matrix corresponding to the first row in the nth convolutional kernel.

2. The method according to claim 1, wherein The obtaining of the weight matrix includes: Extract the i-th row parameters from n m×m convolution kernels, and generate an m×n weight matrix corresponding to the i-th row parameters based on the i-th row parameters, where the number of weight matrices corresponding to the i-th row parameters is m; Load the weight matrix corresponding to the i-th row parameters into the interferometer array of the photonic computing chip, where m, n, and i are all integers greater than or equal to 1.

3. The method according to claim 1, characterized in that, The extracting of the elements belonging to the same row from each output vector and summing them to obtain the convolution calculation result between the convolution kernel and the target feature map includes: Extract the elements belonging to the same row from m output vectors and sum them to obtain the output values corresponding to the m output vectors; Determine the output values corresponding to the m output vectors as the convolution calculation result.

4. A convolutional computing device based on a photonic computing chip, characterized in that, including: An optical signal input end and an optical detector output end. The input end and the output end are connected by an interferometer array arranged in cascade, and the interferometer array is loaded with a weight matrix; The input end is used to obtain a target input vector and perform a linear calculation of the vector-weight matrix using the target input vector to obtain an output vector; The output end is used to extract the elements belonging to the same row from each output vector for summation, so as to obtain the convolution calculation result between the convolution kernel and the target feature map; The weight matrix is generated according to a unitary matrix, a diagonal matrix, and the complex conjugate matrix corresponding to the unitary matrix. The dimension of the weight matrix is m×n, the dimension of the unitary matrix is m×m, the dimension of the diagonal matrix is m×n, and the dimension of the complex conjugate matrix is n×n; The number of interferometers corresponding to the unitary matrix in the interferometer array is calculated based on the following formula: , where is the number of interferometers in the interferometer array corresponding to the unitary matrix; The input end is specifically used to obtain the target feature map to be processed; analyze the target feature map by using a preset sliding window to obtain m to-be-processed feature row vectors, where each feature row vector includes m elements; extract the feature row vector of the i-th row, convert the feature row vector into the feature column vector corresponding to the i-th row, and determine the feature column vector as the target input vector; The input end is specifically used to input the feature column vector corresponding to the i-th row into the input end, so that the input end performs a linear multiplication operation with the weight matrix corresponding to the i-th row to obtain the output vector corresponding to the i-th row; repeatedly run the photonic computing chip until m output vectors are obtained, where each output vector includes n elements; Wherein, the calculation formula is as follows: The result of the first row in the formula corresponds to the dot product accumulation of the target input data and the first row of the first convolution kernel, and the result of the n-th row corresponds to the dot product accumulation of the target input data and the first row of the n-th convolution kernel; a 11 、a 12 、a 13 are the feature vectors located in the first row of the target feature map; w 11 、w 12 、w 13 is the weight matrix of the first row in the first convolutional kernel, w 21 、w 22 、w 23 is the weight matrix of the first row in the second convolutional kernel, and so on, w n1 、w n2 、w n3 is the weight matrix corresponding to the first row in the nth convolutional kernel.

5. The device according to claim 4, characterized in that, The number of interferometers in the interferometer array corresponding to the diagonal matrix is calculated based on the following formula: , where is the number of interferometers in the interferometer array corresponding to the diagonal matrix.

6. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus; wherein: The memory is used to store computer programs; The processor is used to execute the method according to any one of claims 1 to 3 by running the programs stored on the memory.

Images