Photon data stream processing method and system for image classification feature extraction stage

Abstract

The invention discloses a photon data stream processing method for an image classification feature extraction stage, which is applied to the feature extraction stage of image classification, and realizes maximum multiplexing of an input feature map by partitioning and mapping the input feature map according to DNN convolutional layer parameters; the method comprises the following steps of: carrying out electro-optical conversion on an input feature block, modulating the input feature block into a multi-wavelength optical signal, and then carrying out wavelength division multiplexing (WDM) one-to-many broadcast distribution, so that full broadcast of the optical signal is realized; optical domain product accumulation operation is executed through the PCU, so that the throughput calculation speed is increased; and finally, obtaining an output characteristic pattern under the combined action of a photoelectric detector and an ADC (Analog to Digital Converter).

Description

Technical Field

[0001] This invention belongs to the field of photonic computing and deep neural network acceleration technology, and more specifically, relates to a photonic data stream processing method and system for the image classification feature extraction stage. Background Technology

[0002] With the rapid development of deep learning, deep neural networks (DNNs) have been widely used in image classification tasks. To meet the requirements of high throughput, low latency, and scalability for model inference, photonic accelerators, which have the advantages of large-scale parallel transmission and low transmission energy consumption, are an important direction for breaking through the performance bottleneck of existing photonic architectures.

[0003] Most existing photon accelerators follow an electron-centric design paradigm, replacing electronic computing units with photonic devices while retaining the hierarchical structure of electronic memory. This results in some significant drawbacks: First, existing photonic accelerators suffer from low input feature map reuse rates. The same input feature map block cannot be "loaded once and reused multiple times / multiple units," often forcing repeated reading and handling, resulting in high energy consumption for storage access. Second, the existing broadcasting methods of photonic accelerators are mostly local broadcasting, which cannot broadcast the optical signal corresponding to each input feature map block to multiple photonic computing units (PCUs) corresponding to multiple input feature map blocks. Therefore, when multiple PCUs share the same input, the system still completes one-to-many sharing by copying or repeatedly taking data, which leads to insufficient convolutional inference processing speed and significant latency. Third, existing photonic accelerators only perform multiplication / local multiply-add in the optical domain, while accumulation and subsequent operators are still completed in the electrical domain. Frequent photoelectric conversion increases losses and cannot support large-scale convolution operations. Summary of the Invention

[0004] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a photonic data stream processing method and system for the image classification feature extraction stage. Its purpose is to solve the technical problems of high energy consumption for storage access, insufficient convolution inference processing speed and significant latency, and high energy consumption for photoelectric conversion, which cannot be extended to large-scale convolution calculations in existing photonic accelerators.

[0005] To achieve the above objectives, according to one aspect of the present invention, a photon data stream processing method for the image classification feature extraction stage is provided, comprising the following steps: (1) Obtain the total number of convolutional layers in the convolutional neural network and set the convolutional layer counter i=1; (2) Determine whether i is greater than the total number of convolutional layers in the convolutional neural network obtained in step (1). If yes, the process ends; otherwise, proceed to step (3). (3) Obtain the input feature map from DRAM, and process the obtained input feature map using the block division and mapping strategy to obtain multiple input feature map blocks corresponding to the i-th convolutional layer in the convolutional neural network, and the addresses of multiple PCUs corresponding to each input feature map block of the i-th convolutional layer; (4) Perform electro-optical conversion and WDM optical interconnect broadcasting processing on each input feature map block corresponding to the i-th convolutional layer in the convolutional neural network obtained in step (3) to obtain the optical signal corresponding to the input feature map block, and send the optical signal corresponding to the input feature map block to the required input feature map block. Each of the PCUs; (5) Perform multiplication and summation MAC operation on all optical signals corresponding to each input feature map block of the convolutional neural network obtained in step (4) to obtain the optical signal corresponding to the output feature map block of the input feature map block; (6) Perform photoelectric conversion and analog-to-digital conversion on the optical signal corresponding to each output feature block of each input feature map corresponding to the i-th convolutional layer in the convolutional neural network obtained in step (5) to obtain the digital signal corresponding to the output feature block; (7) Write all the digital signals corresponding to all the output feature map blocks of each input feature map in the i-th convolutional layer obtained in step (6) back to DRAM to integrate all the digital signals and obtain the output feature map corresponding to the input feature map, and set the obtained output feature map group as the input feature map; (8) Set the convolutional layer counter i=i+1 and return to step (2).

[0006] Preferably, step (3) includes the following sub-steps: (3-1) Obtain the parameters of the i-th convolutional layer in the convolutional neural network from DRAM, including: an input feature map of size H×L×C, D convolutional kernels of size K×K×C, stride S, padding parameter P, an output feature map of size X×Y×D, and an on-chip photonic computing unit (PCU) of size Z×Z, where H and L represent the height and width of the input feature map, C represents the number of channels in the input feature map, D represents the number of convolutional kernels, and X and Y represent the height and width of the output feature map, respectively, and X... Y ; (3-2) Obtain the height of the output feature map block based on the parameters of the i-th convolutional layer in the convolutional neural network obtained in step (3-1). and width ,in This represents the spatial size of the convolution kernel in the i-th convolutional layer of a convolutional neural network. This represents the number of input channels for the i-th convolutional layer. This represents the bit width of the numerical values ​​in the i-th convolutional layer. The calculated bit width of a single microring resonator MR; (3-3) The height of the output feature map obtained from step (3-2) With width Obtain the height of the input feature map patch of the i-th convolutional layer in a convolutional neural network. ,in Let represent the stride of the i-th convolutional layer. The input feature map is then split according to the height and size of the obtained input feature map blocks to obtain multiple sizes for the i-th convolutional layer in the convolutional neural network. × ×C input feature map patch; (3-4) Based on the parameters of the i-th convolutional layer in the convolutional neural network obtained in step (3-1), obtain the number of output points that a single PCU on the chip can independently compute. The number of PCUs required for each input feature map patch And the total number of PCUs required to complete the computation of the i-th convolutional layer in the convolutional neural network. ,in This represents the height of the output feature map of the i-th convolutional layer. This represents the width of the output feature map of the i-th convolutional layer. This represents the number of convolutional kernels in the i-th convolutional layer, i.e., the number of output feature maps, and... The address of each PCU is assigned to each input feature map block of the i-th convolutional layer in the convolutional neural network.

[0007] Preferably, the number of output points A value greater than 0 indicates that all multiplication, accumulation, and addition operations must be completed within a single PCU, and the number of output points is [value missing]. =0 indicates that all multiplication, accumulation, and addition operations require the collaborative processing of multiple PCUs; exist When >0, =1.

[0008] Preferably, step (4) includes the following sub-steps: (4-1) Obtain the input feature map block obtained in step (3-3), and perform electro-optic conversion processing on each input feature map block through a laser source to obtain the optical signal corresponding to the input feature map block; (4-2) Modulate the optical signal corresponding to each input feature patch to obtain Path wavelength optical signal; (4-3) will The wavelength optical signal is transmitted via WDM through the same optical waveguide to the required path to each input feature block. Each of the PCUs.

[0009] Preferably, step (5) includes the following sub-steps: (5-1) Obtain the kernel weights of the i-th convolutional layer in the convolutional neural network from DRAM, and map the kernel weights to the modulation parameters of the MR array; (5-2) Obtain the optical signal corresponding to each input feature map block of the i-th convolutional layer in the convolutional neural network, and modulate the optical signal according to the modulation parameters obtained in step (5-1) to obtain the weighted optical signal corresponding to the input feature map block; (5-3) The weighted optical signals corresponding to each input feature map block of the convolutional neural network obtained in step (5-2) are superimposed in the optical domain to obtain the optical signals corresponding to the output feature map block of the input feature map block.

[0010] Preferably, in the number of output points When the value is greater than 0, the optical signal obtained in step (5-3) is a complete optical signal; When the number of output points When =0, the optical signal obtained in step (5-3) is a partial and PSUM optical signal.

[0011] Preferably, step (6) includes the following sub-steps: (6-1) Determine the number of output points that a single PCU on the chip can independently calculate. Is it greater than 0 or equal to 0? If it is greater than 0, proceed to step (6-2); otherwise, proceed to step (6-3). (6-2) Use a photodetector to perform photoelectric conversion on the optical signal corresponding to each output feature block obtained in step (5) to obtain an electrical signal, and perform analog-to-digital conversion (ADC) processing on the obtained electrical signal to obtain the digital signal corresponding to the output feature block, and then the process ends. (6-3) Use a photodetector to perform photoelectric conversion on the optical signal corresponding to each output feature block obtained in step (5) to obtain multiple electrical signals. Sum all the electrical signals to obtain the electrical signal accumulation result. Perform ADC processing on the electrical signal accumulation result to obtain multiple digital signals. Sum all the digital signals to obtain the digital signal corresponding to the output feature block.

[0012] According to another aspect of the present invention, a photonic data stream processing system for the image classification feature extraction stage is provided, comprising: The first module is used to obtain the total number of convolutional layers in the convolutional neural network and set the convolutional layer counter i=1; The second module is used to determine whether i is greater than the total number of convolutional layers in the convolutional neural network obtained from the first module. If it is, the process ends; otherwise, it proceeds to the third module. The third module is used to obtain input feature maps from DRAM, process the obtained input feature maps using a block division and mapping strategy, so as to obtain multiple input feature map blocks corresponding to the i-th convolutional layer in the convolutional neural network, and the addresses of multiple PCUs corresponding to each input feature map block of the i-th convolutional layer. The fourth module performs electro-optical conversion and WDM optical interconnect broadcasting processing on each input feature map block corresponding to the i-th convolutional layer in the convolutional neural network obtained by the third module, in order to obtain the optical signal corresponding to the input feature map block, and sends the optical signal corresponding to the input feature map block to the required optical signal of the input feature map block. Each of the PCUs; The fifth module is used to perform multiplication and summation MAC operations on all optical signals corresponding to each input feature map block of the i-th convolutional layer in the convolutional neural network obtained by the fourth module, so as to obtain the optical signal corresponding to the output feature map block of the input feature map block. The sixth module is used to perform photoelectric conversion and analog-to-digital conversion on the optical signal corresponding to each output feature map block of each input feature map corresponding to the i-th convolutional layer in the convolutional neural network obtained by the fifth module, so as to obtain the digital signal corresponding to the output feature map block. The seventh module is used to write back all digital signals corresponding to all output feature map blocks of each input feature map in the i-th convolutional layer obtained by the sixth module to DRAM, so as to integrate and process all digital signals, obtain the output feature map corresponding to the input feature map, and set the obtained output feature map group as the input feature map. The eighth module is used to set the convolutional layer counter i=i+1 and return to the second module.

[0013] In general, the technical solutions and means constructed by this invention can achieve the following beneficial effects: (1) Since the present invention adopts steps (3-3) to (3-4), it calculates the size of the input feature map and organizes the input feature map into blocks, and further completes the mapping and scheduling relationship of "input feature map - PCU" to obtain the input feature map. Therefore, it can solve the technical problem of high energy consumption for storage access in existing photonic accelerators.

[0014] (2) Since the present invention adopts steps (4-1) to (4-3), it completes electro-optic conversion and performs multi-wavelength modulation on the input feature block, and then distributes the corresponding optical signal to the required PCU through WDM to obtain the optical interconnect transmission and distribution path. Therefore, it can solve the technical problems of insufficient convolution inference processing speed and significant delay in existing photonic accelerators. (3) Since the present invention employs steps (5-3) to (6-3), it performs multiplication and addition operations in the optical domain as much as possible to obtain the results corresponding to the output feature blocks. Therefore, it can solve the technical problem that existing photonic accelerators have high photoelectric conversion energy consumption and cannot be extended to large-scale convolution calculation. Attached Figure Description

[0015] Figure 1 This is a flowchart of the photon data stream processing method for the image classification feature extraction stage of this invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0017] The basic idea of ​​this invention is to propose a photonic data stream processing method for the feature extraction stage of image classification. This invention is applied to the feature extraction stage of image classification. By dividing and mapping an input feature map according to the parameters of the DNN convolutional layer, the maximum reuse of the input feature map is achieved. By electro-optically converting and modulating the input feature map blocks into multi-wavelength optical signals and then performing wavelength division multiplexing (WDM) one-to-many broadcast distribution, the full broadcast of the optical signals is achieved. The throughput calculation speed is improved by performing optical domain multiplication and accumulation operations through PCU. Finally, the output feature map is obtained under the joint action of photodetector and ADC.

[0018] like Figure 1 As shown, this invention provides a photon data stream processing method for the image classification feature extraction stage, comprising the following steps: (1) Obtain the total number of convolutional layers in the convolutional neural network and set the convolutional layer counter i=1; (2) Determine whether i is greater than the total number of convolutional layers in the convolutional neural network obtained in step (1). If yes, the process ends; otherwise, proceed to step (3). (3) Obtain the input feature map from the Dynamic Random Access Memory (DRAM), and process the obtained input feature map using the block division and mapping strategy to obtain multiple input feature map blocks corresponding to the i-th convolutional layer in the convolutional neural network, and the addresses of multiple PCUs corresponding to each input feature map block of the i-th convolutional layer. This step includes the following sub-steps: (3-1) Obtain the parameters of the i-th convolutional layer in the convolutional neural network from DRAM, including: an input feature map of size H×L×C, D convolutional kernels of size K×K×C, stride S, padding parameter P, an output feature map of size X×Y×D, and an on-chip photonic computing unit (PCU) of size Z×Z, where H and L represent the height and width of the input feature map, C represents the number of channels of the input feature map, D represents the number of convolutional kernels, and X and Y represent the height and width of the output feature map, respectively, and X... Y ; (3-2) Obtain the height of the output feature map block based on the parameters of the i-th convolutional layer in the convolutional neural network obtained in step (3-1). and width ,in This represents the spatial size of the convolution kernel in the i-th convolutional layer of a convolutional neural network. This represents the number of input channels for the i-th convolutional layer. This represents the bit width of the numerical values ​​in the i-th convolutional layer. This represents the calculated bit width of a single microring resonator (MR). (3-3) The height of the output feature map obtained from step (3-2) With width Obtain the height of the input feature map patch of the i-th convolutional layer in a convolutional neural network. ,in Let represent the stride of the i-th convolutional layer. The input feature map is then split according to the height and size of the obtained input feature map blocks to obtain multiple sizes for the i-th convolutional layer in the convolutional neural network. × ×C input feature map patch.

[0019] (3-4) Based on the parameters of the i-th convolutional layer in the convolutional neural network obtained in step (3-1), obtain the number of output points that can be independently computed by a single PCU on-chip. (among them) A value greater than 0 indicates that all multiplication, accumulation, and addition operations are performed within a single PCU. =0 indicates that multiple PCUs are needed to complete the processing (the number of PCUs required for each input feature map block). (among them) When >0, =1), and the total number of PCUs required to complete the computation of the i-th convolutional layer in the convolutional neural network. ,in This represents the height of the output feature map of the i-th convolutional layer. This represents the width of the output feature map of the i-th convolutional layer. This represents the number of convolutional kernels in the i-th convolutional layer, i.e., the number of output feature maps, and... The address of each PCU is assigned to each input feature map block of the i-th convolutional layer in the convolutional neural network.

[0020] The advantage of steps (3-3) to (3-4) above is that it enables the input feature map to be divided into blocks and complete the "feature map block - PCU" mapping, thereby reducing the energy consumption of storage access.

[0021] (4) Perform electro-optical conversion and WDM optical interconnect broadcasting processing on each input feature map block corresponding to the i-th convolutional layer in the convolutional neural network obtained in step (3) to obtain the optical signal corresponding to the input feature map block, and send the optical signal corresponding to the input feature map block to the required input feature map block. Each of the PCUs; This step includes the following sub-steps: (4-1) Obtain the input feature map block obtained in step (3-3), and perform electro-optic conversion processing on each input feature map block through a laser source to obtain the optical signal corresponding to the input feature map block; (4-2) Modulate the optical signal corresponding to each input feature patch to obtain Path wavelength optical signal; (4-3) will The wavelength optical signal is transmitted via WDM through the same optical waveguide to the required path to each input feature block. Each of the PCUs; The advantage of steps (4-1) to (4-3) above is that the input feature map blocks are synchronously distributed to the required PCUs through electro-optic conversion and WDM, thereby improving the convolutional inference processing speed and reducing latency.

[0022] (5) Perform multiply-accumulate (MAC) operation on all the optical signals corresponding to each input feature map block of the i-th convolutional layer in the convolutional neural network obtained in step (4) to obtain the optical signal corresponding to the output feature map block of the input feature map block; This step includes the following sub-steps: (5-1) Obtain the kernel weights of the i-th convolutional layer in the convolutional neural network from DRAM, and map the kernel weights to the modulation parameters of the MR array (the modulation parameters are reused as the convolutional window traverses all positions of the input feature map). (5-2) Obtain the optical signal corresponding to each input feature map block of the i-th convolutional layer in the convolutional neural network, and modulate the optical signal according to the modulation parameters obtained in step (5-1) to obtain the weighted optical signal corresponding to the input feature map block; (5-3) The weighted optical signals corresponding to each input feature patch in the convolutional neural network obtained in step (5-2) are superimposed in the optical domain to obtain the optical signal corresponding to the output feature patch of that input feature patch (where in When the value is greater than 0, the obtained optical signal is a complete optical signal; When =0, the obtained optical signal is a partial-sum (PSUM) optical signal.

[0023] The advantage of steps (5-1) to (5-3) above is that it enables the PCU to form the result / partial sum by superposition of optical domains, reducing unnecessary photoelectric and analog-to-digital conversions, thereby reducing conversion energy consumption and improving the scalability of large-scale convolutional computation.

[0024] (6) Perform photoelectric conversion and analog-to-digital conversion on the optical signal corresponding to each output feature block of each input feature map corresponding to the i-th convolutional layer in the convolutional neural network obtained in step (5) to obtain the digital signal corresponding to the output feature block; This step includes the following sub-steps: (6-1) Determine the number of output points that a single PCU on the chip can independently calculate. Is it greater than 0 or equal to 0? If it is greater than 0, proceed to step (6-2); otherwise, proceed to step (6-3). (6-2) Use a photodetector to perform photoelectric conversion on the optical signal corresponding to each output feature block obtained in step (5) to obtain an electrical signal, and perform analog-to-digital conversion (ADC) on the obtained electrical signal to obtain the digital signal corresponding to the output feature block, and then the process ends. (6-3) Use a photodetector to perform photoelectric conversion on the optical signal corresponding to each output feature block obtained in step (5) to obtain multiple electrical signals. Sum all the electrical signals to obtain the electrical signal accumulation result. Perform ADC processing on the electrical signal accumulation result to obtain multiple digital signals. Sum all the digital signals to obtain the digital signal corresponding to the output feature block.

[0025] (7) Write all the digital signals corresponding to all the output feature map blocks of each input feature map in the i-th convolutional layer obtained in step (6) back to DRAM to integrate all the digital signals and obtain the output feature map corresponding to the input feature map, and set the obtained output feature map group as the input feature map; (8) Set the convolutional layer counter i=i+1 and return to step (2).

[0026] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A photonic data stream processing method for the image classification feature extraction stage, characterized in that, Includes the following steps: (1) Obtain the total number of convolutional layers in the convolutional neural network and set the convolutional layer counter i=1; (2) Determine whether i is greater than the total number of convolutional layers in the convolutional neural network obtained in step (1). If yes, the process ends; otherwise, proceed to step (3). (3) Obtain the input feature map from DRAM, and process the obtained input feature map using the block division and mapping strategy to obtain multiple input feature map blocks corresponding to the i-th convolutional layer in the convolutional neural network, and the addresses of multiple PCUs corresponding to each input feature map block of the i-th convolutional layer; (4) Perform electro-optical conversion and WDM optical interconnect broadcasting processing on each input feature map block corresponding to the i-th convolutional layer in the convolutional neural network obtained in step (3) to obtain the optical signal corresponding to the input feature map block, and send the optical signal corresponding to the input feature map block to the required input feature map block. Each of the PCUs; (5) Perform multiplication and summation MAC operation on all optical signals corresponding to each input feature map block of the convolutional neural network obtained in step (4) to obtain the optical signal corresponding to the output feature map block of the input feature map block; (6) Perform photoelectric conversion and analog-to-digital conversion on the optical signal corresponding to each output feature block of each input feature map corresponding to the i-th convolutional layer in the convolutional neural network obtained in step (5) to obtain the digital signal corresponding to the output feature block; (7) Write all the digital signals corresponding to all the output feature map blocks of each input feature map in the i-th convolutional layer obtained in step (6) back to DRAM to integrate all the digital signals and obtain the output feature map corresponding to the input feature map, and set the obtained output feature map group as the input feature map; (8) Set the convolutional layer counter i=i+1 and return to step (2).

2. The photon data stream processing method for the image classification feature extraction stage according to claim 1, characterized in that, Step (3) includes the following sub-steps: (3-1) Obtain the parameters of the i-th convolutional layer in the convolutional neural network from DRAM, including: an input feature map of size H×L×C, D convolutional kernels of size K×K×C, stride S, padding parameter P, an output feature map of size X×Y×D, and an on-chip photonic computing unit (PCU) of size Z×Z, where H and L represent the height and width of the input feature map, C represents the number of channels in the input feature map, D represents the number of convolutional kernels, and X and Y represent the height and width of the output feature map, respectively, and X... Y ; (3-2) Obtain the height of the output feature map block based on the parameters of the i-th convolutional layer in the convolutional neural network obtained in step (3-1). and width ,in This represents the spatial size of the convolution kernel in the i-th convolutional layer of a convolutional neural network. This represents the number of input channels for the i-th convolutional layer. This represents the bit width of the numerical representation of the i-th convolutional layer. The calculated bit width of a single microring resonator MR; (3-3) The height of the output feature map obtained from step (3-2) With width Obtain the height of the input feature map patch of the i-th convolutional layer in a convolutional neural network. ,in Let represent the stride of the i-th convolutional layer. The input feature map is then split according to the height and size of the obtained input feature map blocks to obtain multiple sizes for the i-th convolutional layer in the convolutional neural network. × ×C input feature map patch; (3-4) Based on the parameters of the i-th convolutional layer in the convolutional neural network obtained in step (3-1), obtain the number of output points that a single PCU on the chip can independently compute. The number of PCUs required for each input feature map patch And the total number of PCUs required to complete the computation of the i-th convolutional layer in the convolutional neural network. ,in This represents the height of the output feature map of the i-th convolutional layer. This represents the width of the output feature map of the i-th convolutional layer. This represents the number of convolutional kernels in the i-th convolutional layer, i.e., the number of output feature maps, and... The address of each PCU is assigned to each input feature map block of the i-th convolutional layer in the convolutional neural network.

3. The photon data stream processing method for the image classification feature extraction stage according to claim 1 or 2, characterized in that, Number of output points A value greater than 0 indicates that all multiplication, accumulation, and addition operations must be completed within a single PCU, and the number of output points is [value missing]. =0 indicates that all multiplication, accumulation, and addition operations require the collaborative processing of multiple PCUs; exist When >0, =1.

4. The photon data stream processing method for the image classification feature extraction stage according to any one of claims 1 to 3, characterized in that, Step (4) includes the following sub-steps: (4-1) Obtain the input feature map block obtained in step (3-3), and perform electro-optic conversion processing on each input feature map block through a laser source to obtain the optical signal corresponding to the input feature map block; (4-2) Modulate the optical signal corresponding to each input feature patch to obtain Path wavelength optical signal; (4-3) will The wavelength optical signal is transmitted via WDM through the same optical waveguide to the required path to each input feature block. Each of the PCUs.

5. The photon data stream processing method for the image classification feature extraction stage according to claim 4, characterized in that, Step (5) includes the following sub-steps: (5-1) Obtain the kernel weights of the i-th convolutional layer in the convolutional neural network from DRAM, and map the kernel weights to the modulation parameters of the MR array; (5-2) Obtain the optical signal corresponding to each input feature map block of the i-th convolutional layer in the convolutional neural network, and modulate the optical signal according to the modulation parameters obtained in step (5-1) to obtain the weighted optical signal corresponding to the input feature map block; (5-3) The weighted optical signals corresponding to each input feature map block of the convolutional neural network obtained in step (5-2) are superimposed in the optical domain to obtain the optical signals corresponding to the output feature map block of the input feature map block.

6. The photon data stream processing method for the image classification feature extraction stage according to claim 5, characterized in that, Number of output points When the value is greater than 0, the optical signal obtained in step (5-3) is a complete optical signal; When the number of output points When =0, the optical signal obtained in step (5-3) is a partial and PSUM optical signal.

7. The photon data stream processing method for the image classification feature extraction stage according to claim 6, characterized in that, Step (6) includes the following sub-steps: (6-1) Determine the number of output points that a single PCU on the chip can independently calculate. Is it greater than 0 or equal to 0? If it is greater than 0, proceed to step (6-2); otherwise, proceed to step (6-3). (6-2) Use a photodetector to perform photoelectric conversion on the optical signal corresponding to each output feature block obtained in step (5) to obtain an electrical signal, and perform analog-to-digital conversion (ADC) processing on the obtained electrical signal to obtain the digital signal corresponding to the output feature block, and then the process ends. (6-3) Use a photodetector to perform photoelectric conversion on the optical signal corresponding to each output feature block obtained in step (5) to obtain multiple electrical signals. Sum all the electrical signals to obtain the electrical signal accumulation result. Perform ADC processing on the electrical signal accumulation result to obtain multiple digital signals. Sum all the digital signals to obtain the digital signal corresponding to the output feature block.

8. A photonic data stream processing system for the image classification feature extraction stage, characterized in that, include: The first module is used to obtain the total number of convolutional layers in the convolutional neural network and set the convolutional layer counter i=1; The second module is used to determine whether i is greater than the total number of convolutional layers in the convolutional neural network obtained from the first module. If it is, the process ends; otherwise, it proceeds to the third module. The third module is used to obtain input feature maps from DRAM, process the obtained input feature maps using a block division and mapping strategy, so as to obtain multiple input feature map blocks corresponding to the i-th convolutional layer in the convolutional neural network, and the addresses of multiple PCUs corresponding to each input feature map block of the i-th convolutional layer. The fourth module performs electro-optical conversion and WDM optical interconnect broadcasting processing on each input feature map block corresponding to the i-th convolutional layer in the convolutional neural network obtained by the third module, in order to obtain the optical signal corresponding to the input feature map block, and sends the optical signal corresponding to the input feature map block to the required optical signal of the input feature map block. Each of the PCUs; The fifth module is used to perform multiplication and summation MAC operations on all optical signals corresponding to each input feature map block of the i-th convolutional layer in the convolutional neural network obtained by the fourth module, so as to obtain the optical signal corresponding to the output feature map block of the input feature map block. The sixth module is used to perform photoelectric conversion and analog-to-digital conversion on the optical signal corresponding to each output feature map block of each input feature map corresponding to the i-th convolutional layer in the convolutional neural network obtained by the fifth module, so as to obtain the digital signal corresponding to the output feature map block. The seventh module is used to write back all digital signals corresponding to all output feature map blocks of each input feature map in the i-th convolutional layer obtained by the sixth module to DRAM, so as to integrate and process all digital signals, obtain the output feature map corresponding to the input feature map, and set the obtained output feature map group as the input feature map. The eighth module is used to set the convolutional layer counter i=i+1 and return to the second module.

Images