Optimization method of neural network model in optical calculation
By adjusting the quantization scaling factor in the optical computing system to adapt to the noisy environment, the quantization processing of the neural network model was optimized, the impact of noise on the calculation results was resolved, and the accuracy and stability were improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-30
- Publication Date
- 2026-04-07
AI Technical Summary
The impact of noise in optical computing systems on neural network models has not been effectively addressed, affecting the accuracy of calculation results.
By obtaining the initial quantization scaling factor and noise suppression factor, the quantization scaling factor is adjusted to adapt to the noise environment of the optical computing system, and quantization processing is performed.
This reduces the impact of noise in the optical computing system on the neural network model, improving the accuracy and stability of the calculation results.
Smart Images
Figure CN121809566A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to an optimization method for neural network models in optical computing. Background Technology
[0002] In optical computing, noise may exist due to the nature of the optical computing system, which can affect the calculation results. Summary of the Invention
[0003] This invention provides an optimization method for neural network models in optical computing, aiming to solve the problem of how to reduce the impact of noise in optical computing systems on neural network models.
[0004] In a first aspect, the present invention provides a method for optimizing a neural network model in optical computing, the method comprising:
[0005] Obtain the initial quantization scaling factor;
[0006] Obtain the noise suppression coefficient associated with the optical computing system;
[0007] The initial quantization scaling factor is adjusted based on the noise suppression factor to obtain the adjusted quantization scaling factor;
[0008] The neural network model is quantized using the adjusted quantization scaling factor.
[0009] This invention provides an optimization method for neural network models in optical computing, aiming to reduce the impact of optical computing system noise on the neural network model. The method first obtains an initial quantization scaling factor, then acquires a noise suppression factor related to the optical computing system, which reflects the characteristics of noise in the optical computing system. The initial quantization scaling factor is then adjusted based on the noise suppression factor to obtain an adjusted quantization scaling factor. Since the adjusted quantization scaling factor takes into account the noise situation of the optical computing system, when using it to quantize the neural network model, the model can adapt to the noise environment of the optical computing system during the quantization process. Attached Figure Description
[0010] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0011] Figure 1 This is a flowchart of the optimization method for neural network models in optical computing provided by the present invention;
[0012] Figure 2 This is a flowchart of quantization-dequantization without considering noise;
[0013] Figure 3 This is a flowchart of the calculation of the activated noise suppression coefficient provided in an embodiment of the present invention;
[0014] Figure 4 This is a flowchart of the calculation of the noise suppression coefficient of the weights provided in the embodiments of the present invention;
[0015] Figure 5 This is a schematic diagram of optical computing provided in an embodiment of the present invention;
[0016] Figure 6 This is a flowchart of the optimization method for neural network models in optical computing provided in this embodiment of the invention. Detailed Implementation
[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0018] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein.
[0019] In this article, the terms vector and matrix refer to tensors. Tensors can have one-dimensional form (vector), two-dimensional form (matrix), or three-dimensional or higher. Generally, tensors with three or more dimensions are no longer named matrices, but for convenience in this article, when referring to the shape of a matrix, it refers to the shape of a three-dimensional tensor, in the context of the text, without causing confusion. In addition, vectors can exist as a special case of matrices.
[0020] The following combination Figures 1-6 This invention describes an optimization method for a neural network model in optical computing.
[0021] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the optimization method for a neural network model in optical computing provided by the present invention. The optimization method for a neural network model in optical computing includes:
[0022] S110, obtain the initial quantization scaling factor.
[0023] S120, obtains the noise suppression coefficient related to the optical computing system.
[0024] S130, adjust the initial quantization scaling factor according to the noise suppression factor to obtain the adjusted quantization scaling factor.
[0025] S140 uses the adjusted quantization scaling factor to quantize the neural network model.
[0026] The following is a detailed description of steps S110 to S140.
[0027] In some embodiments, step S110, obtaining the initial quantization scaling factor includes:
[0028] S111, determine the initial quantization scaling factor based on the maximum and minimum values of the floating-point numbers corresponding to the data to be quantized.
[0029] Specifically, quantization includes symmetric quantization and asymmetric quantization. For example, the quantization described above can be performed in the following manner, and correspondingly, the initial quantization scaling factor can be obtained from the following "quantization scaling factor":
[0030] (1) Symmetric quantization:
[0031] Symmetric quantization is the process of converting floating-point numbers to values within the range of [-2]. n-1 -2 n-1 A signed integer (integral numeric type, INT) of [-1], where n is the number of bits for symmetric quantization. Assume the floating-point number to be quantized is x. f The quantitative objective is to make x f Perform n-bit symmetric quantization, meaning the range of the quantized fixed-point number is x. q The symmetric quantization process is expressed as the following formula (1).
[0032]
[0033] In formula (1), abs() is the absolute value function, max() is the maximum value function, min() is the minimum value function, round is the rounding function, and x q For a fixed-point number. Furthermore, in formula (1), the Sc term shown in formula (2) below can be called the symmetric quantization scaling factor. That is to say, for a symmetric quantized fixed number, the floating-point number corresponding to that fixed-point number can be determined based on the symmetric quantization scaling factor.
[0034]
[0035] (2) Asymmetric quantization
[0036] Asymmetric quantization is the process of converting floating-point numbers to values in the range [0, 2]. n An unsigned integral numeric type (UINT) number of [-1], where n is the number of bits for asymmetric quantization. Assume the floating-point number to be quantized is x. f The quantitative objective is to make x f Perform n-bit asymmetric quantization, meaning the range of the quantized fixed-point number is [0, 2]. n -1], then the asymmetric quantization process is expressed as the following formula (3).
[0037]
[0038] In formula (3), max() is the function to find the maximum value, min() is the function to find the minimum value, round is the rounding function, and x q For fixed points. Furthermore, in formula (3), as shown in formula (4) below, AS c The term can be called the asymmetric quantization scaling factor, as shown in the following formula (5) Z. p The term can be called an asymmetric quantization zero. That is to say, for an asymmetric quantized fixed-point number, the corresponding floating-point number can be determined based on the asymmetric quantization scaling factor and the asymmetric quantization zero.
[0039]
[0040] Z P =round(-AS) c ×min(x f (5)
[0041] Based on formulas (3) to (5), another representation of asymmetric quantization as shown in formula (6) can be obtained:
[0042] x q =round(x f ×AS c +Z P (6)
[0043] To compare the computations of the model with noise suppression and the model without noise suppression, we can compare... Figure 2 and Figure 6 , can be based on Figure 2 First, understand the model without noise suppression coefficients; further explanation will follow. Figure 6 Describe a neural network model in optical computation with noise suppression coefficients.
[0044] in, Figure 2 The diagram illustrates the quantization of the activation matrix X according to the activation quantization scaling factor xS, resulting in a quantized activation matrix qX. The shape of the activation matrix X is (B, M, K), where B represents the batch size, M represents the number of rows in the matrix, and K represents the number of columns in the matrix.
[0045] For example, when the shape of the activation matrix X is (1, 2, 4), it means that the activation matrix qX to be quantized has one batch of data. In each batch, there are two groups of features, and each group has four elements, i.e., 2 rows and 4 columns. For example, the activation matrix X is:
[0046] [[[1.33, 0.72, -1.55, -0.01], [0.62, -0.72, 0.27, 0.11]]], that is
[0047]
[0048] The activation matrix X represents the corresponding batch dimension B=1. The two sublists [1.33, 0.72, -1.55, -0.01] and [0.62, -0.72, 0.27, 0.11] are two groups. Each group has 4 elements. For example, for the group [1.33, 0.72, -1.55, -0.01], 1.33, 0.72, -1.55, and -0.01 are the 4 elements of this group.
[0049] In this example, if we disregard the noise suppression coefficient, the quantized activation matrix X yields qX as follows:
[0050] [[[3, 1, -3, 0], [3, -3, 1, 0]]], that is
[0051]
[0052] The shape of the quantized activation matrix qX is still (1,2,4), which means that the quantization operation did not change the shape and structure of the matrix.
[0053] For example, the activation quantization scaling factor xS is represented in terms of the third dimension (i.e., K dimensions), and the matrix (tensor) of the activation quantization scaling system xS has a shape of (1,2,1).
[0054] For example, quantizing according to the dimension K=1 yields:
[0055] xS = [[[0.52], [0.24]]], that is
[0056] Where 0.52 corresponds to [1.33, 0.72, -1.55, -0.01], and 0.24 corresponds to [0.62, -0.72, 0.27, 0.11].
[0057] The method also includes operations such as performing shape transformation on the activation matrix X to be quantized, specifically including:
[0058] (1) Perform a shape transformation operation on the activation matrix X to be quantized to transform its shape from (B,M,K) to This indicates rounding up; if K is not divisible by Ks, the insufficient part is padded with 0. Here, Ks represents the number of summation objects in the dot product summation unit of the optical computing hardware (e.g., the optical computing unit). If the summation supported by the summation unit is sum = a1·b1 + a2·b2 + a3·b3, then Ks = 3, where a1·b1 represents the product of numbers a1 and b1 (i.e., the summation object). The meanings of a2·b2 and a3·b3 are similar. Ks = 3 indicates that the summation unit in its hardware is designed to sum 3 objects. If the summation unit's sum is sum = a1·b1 + a2·b2 + a3·b3 + a4·b4, then Ks = 4. Obviously, if the model calculation requires summing more objects, the above shape conversion operation can be used to adapt to the hardware.
[0059] Specifically, in matrix operations, the shape transformation (reshape, or deformation) operation described above is an operation that changes the shape of a matrix. It does not change the element values in the matrix, but only rearranges the layout of these elements. Here, we want to transform the shape of qX from (B, M, K) to...
[0060] For example, given X, with the original shape (1,2,4), when Ks=2,
[0061] Following the rules of the reshape operation, the elements in qX are rearranged. That is, the original X = [[1.33,0.72,-1.55,-0.01],[0.62,-0.72,0.27,0.11]]] becomes [[[1.33,0.72],[-1.55,-0.01]],[0.62,-0.72],[0.27,0.11]]] after shape transformation, with a shape of (2,2,2). Here, the original 1×2×4 matrix is rearranged into a 2×2×2 matrix.
[0062] (2) Perform a matrix transpose operation on the activation matrix X to transform it from... Convert to
[0063] Specifically, the transpose operation rearranges the dimensions of a matrix. Here, it transposes the matrix X after shape transformation. Convert to
[0064] For example, after the previous shape transformation, the shape is (2,2,2). After the transpose operation, the shape is still (2,2,2), but the position of the elements in different dimensions has changed.
[0065] (3) Quantize the transposed activation matrix according to the activation quantization scaling factor xS;
[0066] Here, the activation quantization scaling factor xS corresponding to each activation part can be extracted along the K-dimensional axis. The shape of the activation quantization scaling factor xS is as follows:
[0067] Specifically, this involves obtaining the activation quantization scaling factor xS for each part along K dimensions, and representing it as a shape of... The matrix.
[0068] For example, xS was originally [[[0.52],[0.24]]], with a shape of (1,2,1), and after the operation, it becomes xS=[[[0.52],[0.24]],[[0.52],[0.24]]], with a shape of (2,2,1).
[0069] For example, quantizing the transposed X yields qX = [[[3,1],[3,-3]],[[-3,0],[1,0]]].
[0070] For example, step S112 further includes:
[0071] S1122, Perform a shape transformation operation on the weight matrix W, where the shape of the weight matrix W is (K,N); and quantize the weight matrix W according to the weight quantization scaling factor wS to obtain the quantized weight matrix qW.
[0072] For example, when the shape of the weight matrix W to be quantized is (4,2), it means that the quantized weight matrix qW has 4 rows and 2 columns. Taking W = [[0,-0.17],[0.43,1.20],[-0.97,1.03],[0.23,0.45]] as an example, that is...
[0073]
[0074] Perform a shape transformation operation on the weight matrix W to change its shape from (K,N) to... Where Ks represents the number of summation objects in the dot product summation unit of the optical computing hardware. This indicates rounding up; if K is not divisible by Ks, the missing elements are filled with 0 to represent the corresponding elements.
[0075] Specifically, a shape transformation operation is performed on the weight matrix W, changing its shape from (K,N) to...
[0076] For example, for the above W = [[0,-0.17],[0.43,1.20],[-0.97,1.03],[0.23,0.45]], the shape is (4,2). After performing a shape transformation operation, the shape becomes (2,2,2), and W becomes [[[0,-0.17],[0.43,1.20]],[[-0.97,1.03],[0.23,0.45]]].
[0077] After shape transformation and quantization, we get [[[0,0],[1,3]],[[-3,3],[1,1]]]
[0078] In the quantization process, the weight quantization scaling factor wS can be represented in matrix form. The weight quantization scaling factor wS corresponding to each weight component is extracted along the K-dimensional axis. The shape of this weight quantization scaling factor wS is as follows:
[0079] For example, the shape of the weight quantization scaling factor wS is (2,1,2), and its value is [[[0.32,0.40]],[[0.32,0.40]]].
[0080] In some embodiments, obtaining the noise suppression coefficient related to the optical computing system in step 120 above includes:
[0081] S121, obtain the activation noise suppression coefficient xNSF.
[0082] S122, obtain the weighted noise suppression coefficient wNSF.
[0083] For example, please refer to Figure 3 , Figure 3 This is a flowchart illustrating the calculation of the activation noise suppression coefficient provided in an embodiment of the present invention. Step S121 above includes S1211 to S1214:
[0084] S1211, determine the maximum value maxX and minimum value minX of the quantized activation matrix qX in the Ks dimension. The shape of the quantized activation matrix qX is as follows:
[0085] In other words, the elements of the quantized activation matrix qX are traversed along a specific Ks dimension to find the maximum value (maxX) and minimum value (minx) in each Ks group.
[0086] For example, for [[[3,1],[3,-3]],[[-3,0],[1,0]]], the shapes of maxX and minX are both (2,2,1). Specifically, for [3,1], its minX is [1] and its maxX is [3]. Thus, for [[[3,1],[-3,0]],[[3,-3],[1,0]]], the corresponding minX values are [[[1],[-3]],[[-3],[0]]], and the maxX values are [[[3],[3]],[[0],[1]]].
[0087] S1212, determine the noise suppression basis NFS based on the minimum value minE and the maximum value maxE of the conversion range of the digital-to-analog converter.
[0088] Specifically, please refer to Figure 4 , Figure 4 This is a schematic diagram of the optical computing architecture provided in this embodiment of the invention. The first data and the second data are in digital form, and can be related data of the activation matrix and weight matrix in a neural network model, respectively. The digital-to-analog converter (DAC) converts the digital electrical signal into an analog electrical signal, and generates an optical signal carrying data (input vector) based on the analog electrical signal, i.e., electro-optical conversion. For example, it modulates the optical signal based on the analog electrical signal. The optical computing unit may include a modulator to implement the modulation function. The optical computing unit can process the relevant optical signal and then perform photoelectric conversion, such as photoelectric conversion into an analog electrical signal. The DAC has a minimum value (minE) and a maximum value (maxE) range during conversion. This range is determined by the characteristics of the DAC device itself. For example, different DAC chips can convert different minimum and maximum signal values. Taking a DAC capable of encoding int4 data as an example, the int4 data type represents a signed integer using 4 bits of binary data. In this case, minE is -8 and maxE is 7.
[0089] For example, the maximum value between the absolute value of the minimum value minE and the absolute value of the maximum value maxE can be taken as the noise suppression base NSF.
[0090] In other words, NFS (Noise Suppression Factor) is calculated by taking the maximum of the absolute values of minE and maxE, i.e., NFS = max(abs(minE), abs(maxE)), where abs() is the absolute value function and max() is the maximum value function. For example, in the previous example (minE is -8, maxE is 7), NFS = 8. NFS is related to minE and maxE and can also be called the noise suppression factor.
[0091] S1213, determine the first parameter nNSF based on the minimum value minE and the minimum value minX; and determine the second parameter pNSF based on the maximum value maxE and the maximum value maxX.
[0092] (1) Determine the first parameter nNSF:
[0093] For example, if the minimum value minE is less than 0 and the minimum value minX is also less than 0, then the first parameter nNSF is equal to the result of minE divided by minX and rounded down; otherwise, the first parameter nNSF is equal to the noise suppression base NSF.
[0094] Specifically, when calculating the first parameter nNSF, if the conditions minE < 0 and minX < 0 are met, nNSF is calculated according to the formula. In progress, among which This indicates rounding down to the nearest integer.
[0095] For example, if minE is -4 and minX is -2, then Here, minE is usually less than 0. If minE is greater than or equal to 0, this branch will not be calculated.
[0096] At the same time, the shape of minX is the same as the shape of nNSF, and their positions correspond one-to-one. That is to say, minX is a matrix (with the shape [2,2,1]), and for each element position in minX that is less than 0, the corresponding nNSF position is calculated according to the above formula.
[0097] For example, nNSF = [[[-],[2]],[[2],[-]]], where - indicates no value. This might be because minX does not meet the condition of being less than 0 at this position, so it is not processed according to the given condition. The calculated value was not found.
[0098] If the conditions minE<0 and minX<0 are not met, then nNSF=NSF, NFS=max(abs(minE),abs(maxE)).
[0099] For example, the final result of the first parameter nNSF is [[[8],[2]],[[2],[8]]], with a shape of (2,2,1). This is a three-dimensional matrix with the same shape as the previous minX. This means that each element in nNSF corresponds one-to-one with the element in minX in terms of position.
[0100] (2) Determine the second parameter pNSF:
[0101] For example, if the maximum value maxE is greater than 0 and the maximum value maxX is also greater than 0, then the second parameter pNSF is equal to the result of maxE divided by maxX and rounded down; otherwise, the second parameter pNSF is equal to the noise suppression base NSF.
[0102] Specifically, when calculating the second parameter pNSF, if the conditions maxE>0 and maxX>0 are met, then according to the formula... Perform the calculation.
[0103] For example, if maxE is 4 and maxX is 2, then Here, maxE is usually greater than 0. If maxE is less than or equal to 0, this branch is not calculated.
[0104] The condition maxX>0 means that the pNSF value corresponding to the position in maxX that is greater than 0 is determined according to the formula. Calculation. For example, in the given example pNSF=[[[2],[2]],[[-],[7]]], if maxX does not meet the condition of being greater than 0 at a certain position, - may appear, indicating that there is no value.
[0105] If the conditions maxE>0 and maxX>0 are not met, then pNSF=NSF.
[0106] For example, the final result of the second parameter pNSF is [[[2],[2]],[[8],[7]]], which is also a three-dimensional matrix with a shape of (2,2,1).
[0107] S1214, Based on the first parameter nNSF and the second parameter pNSF, determine the activation noise suppression coefficient xNSF, whose shape is as follows:
[0108] For example, the minimum value between the first parameter nNSF and the second parameter pNSF is taken as the activation noise suppression coefficient xNSF.
[0109] In other words, xNSF = min(nNSF, pNSF), where min() is the minimum function. The shape of xNSF is... The symbol indicates rounding up.
[0110] For example, nNSF = [[[8],[2]],[[2],[8]]], pNSF = [[[2],[2]],[[8],[7]]]. For the calculation of xNSF, for example, at the element position in the first row and first column of xNSF, the value in nNSF is 8, and the value in pNSF is 2. The minimum value is taken using the min() function, so the value of xNSF at this position is 2. Calculating the element at each position in this way, we finally get xNSF = [[[2],[2]],[[2],[7]]], with the shape (2,2,1).
[0111] For example, please refer to Figure 5 , Figure 5 This is a flowchart of the weighted noise suppression coefficient calculation provided in an embodiment of the present invention. In step S122 above, obtaining the weighted noise suppression coefficient wNSF includes steps S1221 to S1224.
[0112] S1221, determine the maximum value maxW and minimum value minW of the quantized weight matrix qW in the Ks dimension. The shape of the quantized weight matrix qW is as follows:
[0113] In other words, the elements of the quantized weight matrix qW are traversed along a specific Ks dimension to find the maximum and minimum values in each Ks group.
[0114] For example, given that the shapes of maxW and minW are both (2,1,1), the value of maxW is [[[1,3]],[[1,3]]], and the value of minW is [[[0,0]],[[-3,1]]].
[0115] S1222, determine the noise suppression basis NFS based on the minimum value minE and the maximum value maxE of the conversion range of the digital-to-analog converter.
[0116] For details, please refer to [link / reference]. Figure 4 The weight vector, also in digital form, needs to be converted into an analog signal by a DAC. The conversion process is similar to that of the input vector. The DAC operates within a range of minimum and maximum values (minE) during conversion. This range is determined by the characteristics of the DAC device itself.
[0117] For example, the maximum value between the absolute value of the minimum value minE and the absolute value of the maximum value maxE can be taken as the noise suppression base NSF.
[0118] In other words, the NFS (Noise Suppression Factor) is calculated by taking the maximum of the absolute values of minE and maxE, i.e., NFS = max(abs(minE), abs(maxE)), where abs() is the absolute value function and max() is the maximum value function. For example, in the previous example (minE is -8, maxE is 7), NFS = 8.
[0119] S1223, determine the first parameter nNSF′ based on the minimum value minE and the minimum value minW; and determine the second parameter pNSF′ based on the maximum value maxE and the maximum value maxW.
[0120] (1) Determine the first parameter nNSF′:
[0121] For example, if the minimum value minE is less than 0 and the minimum value minW is also less than 0, then the first parameter nNSF′ is equal to the result of minE divided by minW and rounded down; otherwise, the first parameter nNSF′ is equal to the noise suppression base NSF.
[0122] Specifically, when calculating the first parameter nNSF′, if minE < 0 and minW < 0, the calculation of nNSF′ follows the formula. conduct, This indicates rounding down. If this condition is not met (i.e., minE≥0 or minW≥0), then nNSF′=NSF.
[0123] For example, the first parameter nNSF′ = [[[8,8]], [[2,8]]].
[0124] (2) Determine the second parameter pNSF′:
[0125] For example, if the maximum value maxE is greater than 0 and the maximum value maxW is also greater than 0, then the second parameter pNSF′ is equal to the result of maxE divided by maxW and rounded down; otherwise, the second parameter pNSF′ is equal to the noise suppression base NSF.
[0126] Specifically, when calculating the second parameter pNSF′, pNSF′ is calculated according to the formula when maxE>0 and maxW>0. Calculate. If this condition is not met (i.e., maxE≤0 or maxW≤0), then pNSF′=NSF.
[0127] For example, the final second parameter pNSF′ = [[[7,2]], [[7,2]]].
[0128] S1224, Based on the first parameter nNSF′ and the second parameter pNSF′, determine the weighted noise suppression coefficient wNSF, whose shape is as follows:
[0129] For example, the minimum value between the first parameter nNSF′ and the second parameter pNSF′ is taken as the weighted noise suppression coefficient wNSF.
[0130] In other words, wNSF = min(nNSF′, pNSF′), where min() is the minimum function. The shape of wNSF is... in It is the floor function, wNSF is a three-dimensional matrix, and the size of the first dimension is 1 / 2. The second dimension has a size of 1, and the third dimension has a size of N.
[0131] For example, nNSF′=[[[8,8]],[[2,8]]],pNSF′=[[[7,2]],[[7,2]]]. Finally, we get wNSF=[[[7,2]],[[2,2]]], with the shape (2,1,2). wNSF is calculated by taking the minimum value from nNSF′ and pNSF′. For example, if the value of nNSF′ at a certain position is 8, and the value of pNSF′ at the corresponding position is 7, then the value of wNSF at that position is 7.
[0132] Please refer to Figure 6 , Figure 6 This is a schematic diagram of an optimization method for a neural network model in optical computing provided by an embodiment of the present invention. Exemplarily, in step 130 above, adjusting the initial quantization scaling factor based on the noise suppression factor to obtain the adjusted quantization scaling factor includes:
[0133] S131, the ratio of the initial quantization scaling factor to the noise suppression factor is used as the adjusted quantization scaling factor. Specifically, this includes:
[0134] S1311, the ratio of the activation quantization scaling factor xS to the activation noise suppression factor xNSF is used as the adjusted activation quantization scaling factor xS′.
[0135] For example, xS = [[[0.52],[0.24]],[[0.52],[0.24]]], with a shape of (2,2,1). xNSF = [[[2],[2]],[[2],[7]]], with a shape of (2,2,1). Since both xS and xNSF have a shape of (2,2,1), xS′ can be calculated by dividing corresponding elements:
[0136] ① For the first row and first column element of xS and the first row and first column element of xNSF:
[0137] The first row and first column element of xS is [0.52], and the first row and first column element of xNSF is [2]. Calculate xS′=xS / xNSF=0.52 / 2=0.26.
[0138] ② For the first row and second column element of xS and the first row and second column element of xNSF:
[0139] The element in the first row and second column of xS is [0.24], and the element in the first row and second column of xNSF is [2]. Calculate xS′=xS / xNSF=0.24 / 2=0.12.
[0140] ③ For the element in the second row and first column of xS and the element in the second row and first column of xNSF:
[0141] The element in the second row and first column of xS is [0.52], and the element in the second row and first column of xNSF is [[2]]. Calculate xS′=xS / xNSF=0.52 / 2=0.26.
[0142] ④ For the second row and second column element of xS and the second row and second column element of xNSF:
[0143] The element in the second row and second column of xS is [[0.24]], and the element in the second row and second column of xNSF is [[7]]. Calculate xS′=xS / xNSF=0.24 / 7≈0.03 (rounded to two decimal places).
[0144] Therefore, xS′=[[[0.26],[0.12]],[[0.26],[0.03]]], with a shape of (2,2,1).
[0145] S1312, the ratio of the weight quantization scaling factor wS to the weight noise suppression factor wNSF is used as the adjusted weight quantization scaling factor wS′.
[0146] For example, wS = [[[0.32,0.40]], [[0.32,0.40]]], with a shape of (2,1,2). wNSF = [[[7,2]], [[2,2]]], with a shape of (2,1,2). Since both wS and wNSF have a shape of (2,1,2), we can calculate wS′ by dividing corresponding elements:
[0147] ① For the first element in the first row and first column of wS and the first element in the first row and first column of wNSF:
[0148] The first element of the first row and first column of wS is 0.32, and the first element of the first row and first column of wNSF is 7. Calculate wS. ′ =0.32 / 7≈0.05 (rounded to two decimal places).
[0149] ② For the second element of the first row and first column of wS and the second element of the first row and first column of wNSF:
[0150] The second element of the first row and first column of wS is 0.40, and the second element of the first row and first column of wNSF is 2. Calculate wS. ′ =0.40 / 2=0.20.
[0151] ③ For the first element of the second row and first column of wS and the first element of the second row and first column of wNSF:
[0152] The first element of the second row and first column of wS is 0.32, and the first element of the second row and first column of wNSF is 2. Calculate wS. ′ =0.32 / 2 = 0.16.
[0153] ④ For the second element in the second row and first column of wS and the second element in the second row and first column of wNSF:
[0154] The second element of the second row and first column of wS is 0.40, and the second element of the second row and first column of wNSF is 2. Calculate wS. ′ =0.40 / 2=0.20.
[0155] Therefore, wS ′ = [[[0.05,0.20]], [[0.16,0.20]]], with a shape of (2,1,2).
[0156] In some embodiments, the optimization method for the neural network model in optical computing further includes:
[0157] S150 performs matrix multiplication calculations, specifically including S151 and S152:
[0158] S151, quantize the activation matrix X according to the adjusted activation quantization scaling factor xS′ to obtain the quantized activation matrix qX′; and further adjust the weight matrix W according to the adjusted weight quantization scaling factor wS′ to obtain the quantized weight matrix qW′.
[0159] S152, perform matrix multiplication on the quantized activation matrix qX′ and the quantized weight matrix qW′ to obtain the matrix multiplication result qO, which has the following shape:
[0160] For example, qX′=[[[3,1],[3,-3]],[[-3,0],[1,0]]], has a shape of (2,2,2); qW′=[[[0,0],[1,3]],[[-3,3],[1,1]]], has a shape of (2,2,2). Then, according to qO=qX′·qW′=[[[1,3],[-3,-9]],[[9,-9],[-3,3]]], its shape is (2,2,2).
[0161] In this way, the neural network model in optical computing can be optimized by optimizing the quantization process.
[0162] In some embodiments, this application also provides a method (step) for dequantization of a neural network model in optical computing, the steps including performing dequantization processing using the optimization method for the neural network model in optical computing described in any of the above embodiments.
[0163] For example, the method for inverse quantization of the neural network model in optical computing includes:
[0164] S160, perform element-wise multiplication with broadcast on the adjusted activation quantization scaling factor xS′ and the adjusted weight quantization scaling factor wS′, i.e., dS = xS′·wS′, to obtain the inverse quantization scaling factor dS, which has the following shape.
[0165] The dequantization scaling factor dS is obtained through matrix multiplication xS′ and wS′, i.e., dS = xS′·wS′, where · denotes point-to-point multiplication with broadcast. The shape of the dequantization matrix is...
[0166] In some embodiments, the inverse quantization of the neural network model in optical computing further includes:
[0167] S170, multiply the dequantization scaling factor dS and the matrix multiplication result qO to obtain the dequantization result O.
[0168] S180 performs cumulative summation on the dequantized result O according to the specified dimension to obtain the dequantized summation result S, which has the shape (1, B*M, N).
[0169] Specifically, the dequantized result O is summed along dimension 0, i.e., the summation result after dequantization is S = O.sum(dim = 0). This means summing the elements of the O matrix along the first dimension. The output shape after summation becomes [1, B*M, N].
[0170] S190 performs a shape transformation operation on the summation result S after dequantization to obtain the final matrix multiplication result R, which has the shape (B, M, N).
[0171] Specifically, the shape of the summation result S after dequantization is finally transformed from [1,B*M,N] to [B,M,N] through the reshape operation.
[0172] The following describes the optical computing neural network model optimization device provided in this application. The optical computing neural network model optimization device described below and the optical computing neural network model optimization method described above can be referred to in correspondence.
[0173] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to ensure that a computer device (which may be a personal computer, server, or network device, etc.) executes the methods described in the various embodiments or some parts of the embodiments.
[0174] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A method for optimizing a neural network model in optical computing, characterized in that, The method includes: Obtain the initial quantization scaling factor; Obtain the noise suppression coefficient associated with the optical computing system; The initial quantization scaling factor is adjusted based on the noise suppression factor to obtain the adjusted quantization scaling factor; The neural network model is quantized using the adjusted quantization scaling factor.
2. The optimization method for neural network models in optical computing according to claim 1, characterized in that, The step of adjusting the initial quantization scaling factor according to the noise suppression factor to obtain the adjusted quantization scaling factor includes: The ratio of the initial quantization scaling factor to the noise suppression factor is used as the adjusted quantization scaling factor.
3. The optimization method for neural network models in optical computing according to claim 1, characterized in that, The process of obtaining the initial quantization scaling factor includes: The initial quantization scaling factor is determined based on the maximum and minimum values of the floating-point numbers corresponding to the data to be quantized.
4. The optimization method for neural network models in optical computing according to claim 1, characterized in that, The method further includes: The quantization process of the neural network model includes: quantizing at least one of the activation matrix and the weight matrix.
5. The optimization method for neural network models in optical computing according to claim 4, characterized in that, This includes obtaining the activation matrix X to be quantized, which has a shape of (B, M, K), where B represents the batch size, M represents the number of rows in the matrix, and K represents the number of columns in the matrix.
6. The method for optimizing a neural network model in optical computing according to claim 5, characterized in that, The method further includes: Perform shape transformation operations on the activation matrix X, including: Perform a shape transformation operation on the activation matrix X to transform its shape from (B,M,K) to... Where Ks represents the number of summation objects in the dot product summation unit of the optical computing hardware. This indicates rounding up; if K is not divisible by Ks, the missing elements are filled with 0; Perform a matrix transpose operation on the activation matrix X to change its shape from Convert to 7. The method for optimizing a neural network model in optical computing according to claim 5, characterized in that, The method further includes: Extract the initial quantization scaling factor xS corresponding to each activation part along the Ks dimension, and define the representation layer shape of the initial quantization scaling factor xS of the activation as follows: The matrix.
8. The method for optimizing a neural network model in optical computing according to claim 4, characterized in that, This includes obtaining the weight matrix W to be quantized: The weight matrix W has a shape of (K, N), where K represents the number of rows in the matrix and N represents the number of columns in the matrix.
9. The method for optimizing a neural network model in optical computing according to claim 8, characterized in that, The method further includes: Perform a shape transformation operation on the weight matrix W to change its shape from (K,N) to... Where Ks represents the number of summation objects in the dot product summation unit of the optical computing hardware. This indicates rounding up; if K is not divisible by Ks, the missing elements are filled with 0.
10. The method for optimizing a neural network model in optical computing according to claim 9, characterized in that, The method further includes: Extract the initial quantization scaling factor wS corresponding to each weight along the K-dimensional axis, and represent the initial quantization scaling factor wS of the weight in the form of... The matrix.
11. The method for optimizing a neural network model in optical computing according to claim 1, characterized in that, The obtained noise suppression coefficient related to the optical computing system includes: The noise suppression basis NFS is determined based on the minimum value minE and the maximum value maxE of the conversion range of the digital-to-analog converter in the optical computing system. Obtain the maximum value maxX and minimum value minX of the activation matrix X in the Ks dimension, where the shape of the activation matrix X is... The first parameter nNSF is determined based on the minimum values minE and minX; and the second parameter pNSF is determined based on the maximum values maxE and maxX. Based on the first parameter nNSF and the second parameter pNSF, the activated noise suppression coefficient xNSF is determined, and its shape is as follows:
12. The method for optimizing a neural network model in optical computing according to claim 1, characterized in that, The obtained noise suppression coefficient related to the optical computing system includes: Obtain the maximum value maxW and minimum value minW of the weight matrix W in the Ks dimension, where the shape of the weight matrix W is as follows: The noise suppression base NFS is determined based on the minimum value minE and the maximum value maxE of the conversion range of the digital-to-analog converter. The first parameter nNSF′ is determined based on the minimum values minE and minW; and the second parameter pNSF′ is determined based on the maximum values maxE and maxW. Based on the first parameter nNSF′ and the second parameter pNSF′, the noise suppression coefficient wNSF of the weights is determined, and its shape is as follows:
13. The method for optimizing a neural network model in optical computing according to claim 10 or 11, characterized in that, The absolute value of the minimum value minE and the absolute value of the maximum value maxE are taken as the noise suppression base NSF.
14. The method for optimizing a neural network model in optical computing according to claim 10 or 11, characterized in that, If the minimum value minE is less than 0 and the minimum value minX is also less than 0, then the first parameter nNSF is equal to the result of minE divided by minX and rounded down; otherwise, the first parameter nNSF is equal to the noise suppression base NSF. If the minimum value minE is less than 0 and the minimum value minW is also less than 0, then the first parameter nNSF′ is equal to the result of minE divided by minW and rounded down; otherwise, the first parameter nNSF′ is equal to the noise suppression base NSF.
15. The method for optimizing a neural network model in optical computing according to claim 10 or 11, characterized in that, If the maximum value maxE is greater than 0 and the maximum value maxX is also greater than 0, then the second parameter pNSF is equal to the result of maxE divided by maxX and rounded down; otherwise, the second parameter pNSF is equal to the noise suppression base NSF. If the maximum value maxE is greater than 0 and the maximum value maxW is also greater than 0, then the second parameter pNSF′ is equal to the result of maxE divided by maxW and rounded down; otherwise, the second parameter pNSF′ is equal to the noise suppression base NSF.
16. The method for optimizing a neural network model in optical computing according to claim 10 or 11, characterized in that, The minimum value between the first parameter nNSF and the second parameter pNSF is taken as the activation noise suppression coefficient xNSF; The minimum value between the first parameter nNSF′ and the second parameter pNSF′ is taken as the noise suppression coefficient wNSF for the weight.
17. The method for optimizing a neural network model in optical computing according to claim 2, characterized in that, The step of using the ratio of the initial quantization scaling factor to the noise suppression factor as the adjusted quantization scaling factor includes: The ratio of the initial quantization scaling factor xS to the noise suppression factor xNSF is used as the adjusted activation quantization scaling factor xS′. The ratio of the initial quantization scaling factor wS to the noise suppression factor wNSF of the weight is used as the adjusted weight quantization scaling factor wS′.
18. The method for optimizing a neural network model in optical computing according to claim 17, characterized in that, The step of quantizing the neural network model using the adjusted quantization scaling factor further includes: Based on the adjusted activation quantization scaling factor xS′, the activation matrix X to be quantized is quantized to obtain the quantized activation matrix qX′; and based on the adjusted weight quantization scaling factor wS′, the weight matrix qW is quantized to obtain the quantized weight matrix qW′. Perform matrix multiplication on the quantized activation matrix qX′ and the quantized weight matrix qW′ to obtain the matrix multiplication result qO, which has the following shape:
19. The optimization method for neural network models in optical computing according to claim 18, characterized in that, The method further includes: The adjusted activation quantization scaling factor xS′ and the adjusted weight quantization scaling factor wS′ are multiplied element-wise with broadcast, i.e., dS = xS′·wS′, to obtain the inverse quantization scaling factor dS, which has the following shape. Multiply the dequantization scaling factor dS and the matrix multiplication result qO to obtain the dequantized result O.
20. The method according to claim 19, characterized in that, The method further includes: The dequantized result O is summed according to the specified dimension to obtain the dequantized summation result S, which has the shape (1, B*M, N). Perform a shape transformation operation on the summation result S after dequantization to obtain the final matrix multiplication result R, which has the shape (B, M, N).