Method for accelerating FCNN training in on-chip optical network of Mesh structure

Abstract

The invention provides a method for accelerating FCNN training in an on-chip optical network of a mesh structure. The invention discloses a scheme for accelerating full-connection neural network (FCNN) training in an on-chip optical network (ONoC) of a mesh topological structure. According to the acceleration scheme, an FCNN acceleration model of ONoC based on mesh is provided; the method comprises the following steps: firstly, designing an FCNN perceptual mapping strategy, and mapping neurons in the FCNN to a core on an ONoC; a packet inter-core communication scheme with a low wavelength requirement is provided according to the distribution of the mapping cores; then, the communication time and the calculation time are balanced, so that the optimal kernel number in each period is obtained;

Description

Technical Field

[0001] The present invention belongs to the field of intersection of on-chip optical networks and neural networks, and specifically relates to a method for accelerating FCNN training in an on-chip optical network with a Mesh structure. Background Art

[0002] In recent years, the acceleration of the neural network training process has attracted much attention. Usually, the training of neural networks is done on CPU / GPU. However, with the explosive growth of network size, a single CPU / GPU seems to be insufficient to meet the needs of the network. To this end, researchers have designed different platforms to accelerate neural networks in the past few decades. These acceleration schemes are mainly divided into three categories: application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and mapping strategies. ASICs are designed for specific applications and provide some solutions for specific deep neural network models, but this design lacks flexibility, is not reconfigurable, performs poorly under different applications, and incurs huge design costs. Due to the programmable nature of FPGAs, it is more flexible than ASIC-based deep neural network accelerators, but they are still designed for specific deep neural network models and application configurations. Therefore, there is an urgent need for a method that can improve the computational efficiency and energy efficiency of neural networks, as well as improve flexibility, scalability, and anti-interference. Summary of the invention

[0003] The purpose of the present invention is to solve the technical problems in the above background and propose a method for accelerating FCNN training in an on-chip optical network with a Mesh structure:

[0004] S1, map the neurons in the same layer of FCNN to the same column / row in the on-chip optical network of the Mesh structure;

[0005] S2. According to the distribution of cores in the mapping strategy, an inter-core communication model is constructed and the communication time of each cycle is obtained;

[0006] S3, obtaining the computing time required for FCNN training under the mapping strategy and communication mode in step S1 and step S2;

[0007] S4. By balancing the computing time and communication time, the optimal number of cores for each cycle is obtained.

[0008] In the preferred solution, S21, assuming that there are n cores to be mapped in a column, the n cores are divided into L G 1 Group; Definition G k Group: G k+1 The group contains r G k Group, including r k+1 cores, the general formula is:

[0009] Gk+1 , r k-1 <L≤r k ;

[0010] Where k = 1, 2, 3…, r is G 1 The number of cores included in the group, r = 2λ + 1, λ is the number of available wavelengths;

[0011] S22, determine the central node Q of each column of mapped core i :

[0012]

[0013] S23, in the forward propagation phase, the core of a column mapped to the output layer first collects the data into groups and puts them into the central node of the column, and then transmits all the data to the central node of the next column. The cycle takes:

[0014]

[0015] S24, the central node of the column to which the first hidden layer is mapped sends data to the other mapped cores of the column. After the data calculation is completed, all data packets are collected to the central node and finally transmitted to the central node of the next column. The communication process of all hidden layers is the same. The cycle time corresponding to one hidden layer is:

[0016]

[0017] S25, the central node corresponding to the output layer sends all the data from the last hidden layer to the remaining mapped cores in the column, and the time consumed is:

[0018]

[0019] S26. The communication process in the back propagation phase is exactly the opposite of the forward propagation.

[0020] In the preferred solution, it is assumed that the number of neurons in one layer of FCNN is N i , i∈[0,t], a layer of neurons is evenly mapped to a column of kernels, and the number of neurons mapped to a kernel is Use f1(n i ) represents the nth cycle of the forward propagation stage i The communication time of each core is:

[0021]

[0022] Where B is the wavelength bandwidth, c is the delay caused by optoelectronic conversion or routing decision, and α i represents the amount of data to be processed by each neuron in the i-th stage of the forward propagation process, and the number of steps required in the i-th cycle data collection stage is expressed as W is the data size coefficient, which represents the sum of the data volume transmitted from the first step to the last step. The data volume in the last step is represented by the average value, then:

[0023]

[0024] Use f2(n i ) represents the communication time of the i-th cycle in the back propagation phase, then:

[0025]

[0026] Among them, β represents the amount of data processed by a single neuron during back propagation.

[0027] In the preferred solution, the total communication time T comm It is expressed as:

[0028]

[0029] In the preferred solution, the computation time for each cycle in the forward propagation phase is: In the output layer, the predicted value needs to be compared with the true value, and the appropriate loss function is used to calculate the loss, so the output layer calculation time is:

[0030]

[0031] The computing power of each core in processing data is k, and the amount of computation required for each neuron in the output layer to calculate the loss is p;

[0032] The computation time for each cycle of back propagation is: The amount of computation required to update a weight of each neuron is θ, and the amount of computation required to update the bias parameter is η.

[0033] In the preferred embodiment, g(n i ) means n in i cycle i The computation time of each core is:

[0034]

[0035] in

[0036] In the preferred solution, the total computation time is defined as T comp ,have:

[0037]

[0038] In a preferred embodiment, step S4 comprises the following steps:

[0039] S41. The total duration of one training period is:

[0040] T=T input +T comm +T comp +ξ,

[0041] The time delay caused by loading external input data and FCNN instructions from main memory to the specified core is denoted by T input Denotes,ζ represents the additional delay caused by additional main memory access, software overhead, or synchronization;

[0042] S42, when i = 0, the column mapped to the input layer only involves communication, not calculation, and the number of cores is defined as n p ,have:

[0043] n0=n p ;

[0044] S43, when 1≤i≤t-1, f1(n i )、f2(n 2t-i+1 )、g(n i )、g(n 2t-i+1 ) is n i In order to find the extreme value of the function, we add them together and take the derivative to make them equal to 0, and get the optimal number of cores:

[0045]

[0046] S44, when i = t, f1(n t ), f2(n t+1 ), g(n t ), g(n t+1 ) is n t The functions are:

[0047]

[0048] The beneficial effects of the present invention are as follows: the present invention uses a group-based inter-core communication scheme to effectively complete the communication of FCNN training using limited wavelengths. Then, by achieving a trade-off between computing time and communication time in each cycle of FCNN training, the optimal number of cores for each cycle is obtained, thereby minimizing the training time. The invention can deeply explore the profound connotation of accelerating neural networks using on-chip optical networks, and lay a technical foundation for promoting the development of neural network accelerators based on mapping methods. The simulation results of the present invention show that the scheme is superior to the two traditional communication schemes in the ONoC environment; under the same communication mode, it is superior to the ENoC environment. The scheme has the advantages of low wavelength requirements, short training time, and good scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a schematic diagram of the mapping from FCNN to ONoC.

[0050] Figure 2 Schematic diagram of the inter-core communication path.

[0051] Figure 3 Schematic diagram of the data collection phase when the number of available wavelengths is 2.

[0052] Figure 4 The following is a comparison chart of the training time of the three schemes under different FCNN models and different numbers of available wavelengths.

[0053] Figure 5 Comparison of the training time per cycle of FCNN in ONoC and NoC under different fixed cores.

[0054] Figure 6 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0055] A scheme for accelerating the training of a fully connected neural network in an on-chip optical network with a mesh topology includes the following steps:

[0056] Step 1: Map the neurons in the same layer of FCNN to the same column / row in the mesh structure of ONoC, such as Figure 1 As shown;

[0057] Step 2: According to the distribution of cores in the mapping strategy, construct the inter-core communication model and obtain the communication time of each cycle, such as Figure 2 As shown;

[0058] Step 2 is broken down into the following steps:

[0059] Step 2.1: Assume that there are n mapped cores in a column, and divide the n cores into L G1 groups. To complete the communication more efficiently, define the G2 group: the G2 group can contain r G1 groups and r2 cores. Similarly, the G3 group can contain r G2 groups and r3 cores. Similarly, the general formula of the group is:

[0060] G k+1 ,r k-1 <L≤r k (1)

[0061] Where k = 1, 2, 3…, r is the number of cores contained in group G1, r is determined by the number of available wavelengths λ, r = 2λ + 1;

[0062] Step 2.2: Determine the central node of each column of the mapped core (the first node in each column is labeled 1):

[0063]

[0064] Step 2.3: In the forward propagation (FP) phase, the output layer (i=0) is mapped to a column of kernels that first collect the data in groups (e.g. Figure 3 As shown) to the center node of the column (time Tb i ), and then transfer all data to the central node of the next column (the column of kernels mapped to the first hidden layer) (time Tc i ), the cycle takes:

[0065] Tcomm i =Tb i +Tc i ,i=0 (3)

[0066] Step 2.4: The central node of the column mapped to the first hidden layer sends data to the other mapped cores of the column (time Ta i ), after the data calculation is completed, all data groups are collected to the central node and finally transmitted to the central node of the next column. All hidden layers (i∈[1,t-1]) are this communication process. The cycle time corresponding to one hidden layer is:

[0067] Tcomm i =Ta i +Tb i +Tc i ,i∈[1,t-1] (4)

[0068] Step 2.5: The central node corresponding to the output layer (i=t) sends all the data from the last hidden layer to the remaining mapped cores in the column, and the time taken is:

[0069] Tcomm i =Ta i ,i=t (5)

[0070] Step 2.6: The communication process in the back propagation (BP) phase is exactly the opposite of FP;

[0071] Step 2.7: Assume that the number of neurons in one layer of FCNN is N i , i∈[0,t], a layer of neurons is evenly mapped to a column of cores, one core can be mapped to one or more neurons, then the number of neurons mapped to a core is Use f1(n i ) represents the nth cycle of the FP phase i The communication time of each core is:

[0072]

[0073] Where B represents the wavelength bandwidth, c represents the delay caused by optoelectronic conversion, routing decision, etc., and α irepresents the amount of data to be processed by each neuron in the i-th stage of the forward propagation process, and the number of steps required in the i-th cycle data collection stage is expressed as W is the data size coefficient, which represents the sum of the data volume transmitted from the first step to the last step, where the data volume of the last step is represented by the average value.

[0074] Use f2(n i ) represents the communication time of the i-th segment in the BP phase, then:

[0075]

[0076] Among them, β represents the amount of data processed by a single neuron during the BP process;

[0077] Step 2.8: The total communication time Tcomm is expressed as:

[0078]

[0079] Step 3: Obtain the computational time required for FCNN training under the mapping strategies and communication modes in steps S1 and S2;

[0080] The decomposition steps of step 3 are:

[0081] Step 3.1: The computation time of each cycle in the FP phase is: In the output layer, the predicted value needs to be compared with the true value, and the appropriate loss function is used to calculate the loss, so the output layer calculation time is:

[0082] The computing power of each core in processing data is k, and the amount of computation required for each neuron in the output layer to calculate the loss is p;

[0083] Step 3.2: The calculation time of each BP cycle is:

[0084] The amount of computation required to update a weight value of each neuron is θ, and the amount of computation required to update the bias parameter is η;

[0085] Step 3.3: Define g(n i ) means n in i cycle i The computation time of each core is:

[0086]

[0087] in

[0088] Step 3.4: The total computation time is defined as Tcomp, which is:

[0089]

[0090] Step 4: Determine the optimal number of cores for each cycle by weighing the computing time and communication time;

[0091] The decomposition steps of step 4 are:

[0092] Step 4.1: In neural networks, an epoch refers to the process in which all training data sets are sent to the neural network and FP and BP are completed once; here we only consider the training of one epoch, because each epoch is repeated in the training process of FCNN; the total training time of one epoch is:

[0093] T=Tinput+Tcomm+Tcomp+ξ (11)

[0094] Among them, the time delay caused by loading external input data and FCNN instructions from the main memory to the specified core is represented by Tinput, and ζ represents the additional delay caused by additional main memory access, software overhead, synchronization, etc.

[0095] Step 4.2: When i = 0, the columns mapped to the input layer only involve communication, not computation. To minimize the communication time at this stage, the number of cores used should be as small as possible. On this basis, the system's parallelism requirements (loading multiple input data at the same time), data flow, and load balancing should be considered. Define the number of cores used during this period as n p ,have:

[0096] n0=n p (12)

[0097] Step 4.3: When 1≤i≤t-1, f1(n i )、f2(n 2t-i+1 )、g(n i )、g(n 2t-i+1 ) is n i In order to find the extreme value, we add them together and take their derivatives to make them equal to 0, and get the optimal number of cores:

[0098]

[0099] Step 4.4: When i = t, f1(n t ), f2(n t+1 ), g(n t ), g(n t+1 ) is n i Similarly, we have:

[0100]

[0101] Step 5 is: According to the training time of one epoch of FCNN, two traditional schemes in ONoC are compared with the same communication mode in ENoC, such as Figure 4 and Figure 5 It can be seen that the model of the present invention shortens the training time of FCNN in both ONoC and ENoC, and a good acceleration effect can be achieved by using a small number of wavelengths in ONoC.

[0102] Simulation results show that the scheme has the advantages of low wavelength requirement, short training time and good scalability.

[0103] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for accelerating FCNN training in an on-chip optical network with a Mesh structure, characterized by: S1, map the neurons in the same layer of FCNN to the same column / row in the on-chip optical network of the Mesh structure; S2. According to the distribution of cores in the mapping strategy, an inter-core communication model is constructed and the communication time of each cycle is obtained; S3, obtaining the computing time required for FCNN training under the mapping strategy and communication mode in step S1 and step S2; S4. By balancing the computing time and communication time, the optimal number of cores for each cycle is obtained.

2. According to claim 1, a method for accelerating FCNN training in an on-chip optical network with a Mesh structure is characterized by: Step S2 includes the following steps: S21. Assume that there are n mapped cores in a column, and divide the n cores into L G 1 Group; Definition G k Group: G k+1 The group contains r G k Group, including r k+1 cores, the general formula is: G k+1 ,r k-1 <L≤r k ; Where k = 1, 2, 3…, r is G 1 The number of cores included in the group, r = 2λ + 1, λ is the number of available wavelengths; S22, determine the central node Q of each column of mapped core i : S23, in the forward propagation phase, the core of a column mapped to the output layer first collects the data into groups and puts them into the central node of the column, and then transmits all the data to the central node of the next column. The cycle takes: S24, the central node of the column to which the first hidden layer is mapped sends data to the other mapped cores of the column. After the data calculation is completed, all data packets are collected to the central node and finally transmitted to the central node of the next column. The communication process of all hidden layers is the same. The cycle time corresponding to one hidden layer is: S25, the central node corresponding to the output layer sends all the data from the last hidden layer to the remaining mapped cores in the column, and the time consumed is: S26. The communication process in the back propagation phase is exactly the opposite of the forward propagation.

3. According to claim 2, a method for accelerating FCNN training in an on-chip optical network with a Mesh structure is characterized by: Assume that the number of neurons in one layer of FCNN is N i , i∈[0,t], a layer of neurons is evenly mapped to a column of kernels, and the number of neurons mapped to a kernel is Use f1(n i ) represents the nth cycle of the forward propagation stage i The communication time of each core is: Where B is the wavelength bandwidth, c is the delay caused by optoelectronic conversion or routing decision, and α i represents the amount of data to be processed by each neuron in the i-th stage of the forward propagation process, and the number of steps required in the i-th cycle data collection stage is expressed as W is the data size coefficient, which represents the sum of the data volume transmitted from the first step to the last step. The data volume in the last step is represented by the average value. Then: Use f2(n i ) represents the communication time of the i-th cycle in the back propagation phase, then: Among them, β represents the amount of data processed by a single neuron during back propagation.

4. According to claim 3, a method for accelerating FCNN training in an on-chip optical network with a Mesh structure is characterized by: Total communication time T comm It is expressed as:

5. The method for accelerating FCNN training in an on-chip optical network with a Mesh structure according to claim 2, characterized in that: The computation time for each cycle of the forward propagation phase is: In the output layer, the predicted value needs to be compared with the true value, and the appropriate loss function is used to calculate the loss, so the output layer calculation time is: The computing power of each core in processing data is k, and the amount of computation required for each neuron in the output layer to calculate the loss is p; The computation time for each cycle of back propagation is: The amount of computation required to update a weight of each neuron is θ, and the amount of computation required to update the bias parameter is η.

6. According to claim 5, a method for accelerating FCNN training in an on-chip optical network with a Mesh structure is characterized by: defining g(n i ) means n in i cycle i The computation time of each core is: in 7. The method for accelerating FCNN training in an on-chip optical network with a Mesh structure according to claim 6, characterized in that: The total computation time is defined as T comp ,have:

8. The method for accelerating FCNN training in an on-chip optical network with a Mesh structure according to claim 1, characterized in that: Step S4 includes the following steps: S41. The total duration of one training period is: T=T input +T comm +T comp +ξ, The time delay caused by loading external input data and FCNN instructions from main memory to the specified core is denoted by T input Denotes,ζ represents the additional delay caused by additional main memory access, software overhead, or synchronization; S42, when i = 0, the column mapped to the input layer only involves communication, not calculation, and the number of cores is defined as n p ,have: n0=n p ; S43, when 1≤i≤t-1, f1(n i )、f2(n 2t-i+1 )、g(n i )、g(n 2t-i+1 ) is n i In order to find the extreme value of the function, we add them together and take the derivative to make them equal to 0, and get the optimal number of cores: S44, when i = t, f1(n t ), f2(n t+1 ), g(n t ), g(n t+1 ) is n t The functions are:

Images