Training deep learning model based on characteristics of accelerator to improve energy efficiency in accelerating calculations of model

Abstract

The present disclosure relates to training a deep learning model based on characteristics of an accelerator to improve energy efficiency in accelerating calculations of the model. Based on the type of accelerator to be used to perform calculations of the artificial neural network, a weight matrix of the artificial neural network may be adjusted during training or via retraining based on energy consumption characteristics of the type of accelerator. Weight expressions that may consume more energy in calculations performed via the accelerator may be suppressed via penalty by a loss function during training or via pruning and retraining. The adjusted weight matrix may be configured in a computing device having an accelerator of the type. When the computing device performs calculations of the artificial neural network using the weight matrix, the accelerator may be used to accelerate multiplication and accumulation operations involving the weight matrix.

Description

[0001] Related Applications

[0002] This application claims priority to Provisional U.S. Patent Application No. 63 / 485,470, filed on February 16, 2023, the entire disclosure of which is hereby incorporated by reference herein. Technical Field

[0003] At least some embodiments disclosed herein relate generally to multiply and accumulate computations, and more particularly, but not limited to, reducing energy usage in computations of deep learning models. Background Art

[0004] Many techniques have been developed to accelerate multiplication and accumulation calculations. For example, multiple sets of logic circuits can be configured into arrays to perform multiplication and accumulation in parallel to accelerate multiplication and accumulation operations. For example, photon accelerators have been developed to use phenomena in the optical domain to obtain computational results corresponding to multiplication and accumulation. For example, a memory subsystem can use a memristor crossbar switch or array to accelerate multiplication and accumulation operations in the electrical domain.

[0005] The memory subsystem may include one or more memory devices that store data. The memory devices may be, for example, non-volatile memory devices and volatile memory devices. In general, the host system may utilize the memory subsystem to store data at the memory devices and retrieve data from the memory devices. Summary of the invention

[0006] On the one hand, the present disclosure provides a method, which includes: identifying the type of accelerator for multiplication and accumulation operations; adjusting the weight matrix of an artificial neural network based on the energy consumption characteristics of the type of accelerator; configuring the weight matrix adjusted according to the energy consumption characteristics in a computing device having the type of accelerator; and using the type of accelerator to accelerate the multiplication and accumulation operations in the calculation of the artificial neural network performed using the weight matrix configured in the computing device. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings in which like references indicate similar elements.

[0008] Figure 1 Customization of training of a weight matrix of an artificial neural network based on characteristics of an accelerator used to accelerate computation of the artificial neural network according to one embodiment is described.

[0009] Figure 2 Retraining a weight matrix of an artificial neural network to improve energy efficiency in accelerating computation of the artificial neural network according to one embodiment is described.

[0010] Figure 3 Energy consumption characteristics of some types of accelerators for customized training of artificial neural networks according to some embodiments are presented.

[0011] Figure 4 An analog accelerator implemented using a microring resonator is shown according to one embodiment.

[0012] Figure 5 Another accelerator implemented using a microring resonator according to one embodiment is shown.

[0013] Figure 6 A calculation is shown that multiplies a weight bit column by an input bit column to provide an accumulated result according to one embodiment.

[0014] Figure 7 A calculation is shown that multiplies a multi-bit weight column by an input bit column to provide an accumulated result according to one embodiment.

[0015] Figure 8 A calculation is shown that multiplies a multi-bit weight column by a multi-bit input column to provide an accumulated result according to one embodiment.

[0016] Fig. 9 A processing unit configured to perform matrix-matrix operations is shown according to one embodiment.

[0017] Fig.10 A processing unit configured to perform matrix-vector operations is shown according to one embodiment.

[0018] Fig.11 A processing unit configured to perform vector-vector operations is shown according to one embodiment.

[0019] Fig.12 An example computing system with an accelerator is shown according to one embodiment.

[0020] Fig.13 A method for training a deep learning model according to one embodiment is presented. DETAILED DESCRIPTION

[0021] At least some embodiments disclosed herein provide techniques for reducing energy consumption in computations of deep learning models that can be accelerated using an accelerator for multiplication and accumulation.

[0022] Accelerators for multiplication and accumulation may be implemented via different types of technologies, such as microring resonators, synaptic memory cells, logic circuits, memristors, etc. As a result, the accelerators may have different energy consumption characteristics. A particular type of accelerator may consume less energy and therefore be beneficial in reducing energy consumption when performing computations on inputs having one set of characteristics, but not when performing computations on inputs having another set of characteristics. A deep learning model may be customized at training time to have characteristics that are beneficial in reducing energy consumption when accelerating computations of a trained deep learning model via a particular type of accelerator for multiplication and accumulation.

[0023] Typical deep learning techniques include training a model of an artificial neural network based on a training data set. The training operation is configured to adjust the parameters of the artificial neural network in the form of a weight matrix so that the artificial neural network can produce the desired output as indicated in the training data set in response to the input specified in the training data set.

[0024] In at least some embodiments disclosed herein, training operations are tailored to fine-tune the weight matrix to have properties that are more energy efficient for processing via a particular type of accelerator that will be used to accelerate computations of a model of an artificial neural network.

[0025] For example, when the input data of a task has a large number of values ​​(or can be transformed to have a large number of values, for example, via a bitwise left shift), an accelerator implemented via a microring resonator can consume less energy than other types of accelerators when performing the task. Therefore, when a model of an artificial neural network is to be implemented in a computing device having such an accelerator, the back-propagation phase of the training of the weight matrix of the artificial neural network can be implemented to include a loss function that penalizes small weights. Therefore, the trained weight matrix can have a weight distribution that is more concentrated on large weights than the resulting weight matrix trained without using a loss function. Therefore, when multiplication and accumulation calculations involving weight matrices are accelerated via an accelerator having a microring resonator as a computing element, the training implemented using a loss function can produce a weight matrix with reduced energy consumption.

[0026] For example, when most of the bits of the input data of a task have a value of 0 (or can be transformed to have a majority of 0, for example, via bit reversal), an accelerator implemented via a synaptic memory unit can consume less energy than other types of accelerators when performing the task. Therefore, when a model of an artificial neural network is to be implemented in a computing device having such an accelerator, the back-propagation phase of the training of the weight matrix of the artificial neural network can be implemented as a loss function that includes penalizing weight bits having a value of 1 (or 0, when the weight has more 0 bits than 1 bits). Therefore, the trained weight matrix can have a weight bit distribution with an increased ratio between bits of different values. Therefore, when multiplication and accumulation calculations involving weight matrices are accelerated via an accelerator having a synaptic memory unit as a computing element, the training implemented with the use of a loss function can produce a weight matrix with reduced energy consumption.

[0027] For example, when the input data of a task has a small magnitude (or can be transformed to have a small magnitude, for example, via a bitwise right shift), an accelerator implemented via a memristor can consume less energy than other types of accelerators when performing the task. Therefore, when a model of an artificial neural network is to be implemented in a computing device having such an accelerator, the back-propagation phase of the training of the weight matrix of the artificial neural network can be implemented as a loss function that includes a penalty for large weights. Therefore, compared to a weight matrix trained without using a loss function, the trained weight matrix can have a weight distribution that is more concentrated on small weights. Therefore, when multiplication and accumulation calculations involving weight matrices are accelerated via an accelerator having memristors as computing elements, training implemented using a loss function can produce a weight matrix with reduced energy consumption.

[0028] For example, an accelerator implemented via logic circuits may consume a certain amount of energy that is substantially independent of the weight values ​​in the weight matrix. Therefore, when a model of an artificial neural network is to be implemented in a computing device having such an accelerator, training of the weight matrix of the artificial neural network may be implemented without using a loss function to fine-tune the weights so that the training may achieve improved accuracy.

[0029] Optionally, when a model of an artificial neural network trained without using a loss function (e.g., trained for an accelerator having logic circuits as computing elements) is to be deployed in a computing device having an accelerator with different energy consumption characteristics, the initially trained model can be retrained to implement selective pruning and thereby improve the energy efficiency of the retrained model implemented in the computing device.

[0030] For example, when an accelerator in a computing device is configured with a microring resonator as a computing element, an original model trained using a training data set without using a loss function can be retrained to prune or increase small weights while minimizing the difference between the output of the original model and the output of the retrained model in the retraining operation. Therefore, the retrained model can be accelerated in the computing device with less energy consumption than the original model.

[0031] For example, when an accelerator in a computing device is configured with a memristor as a computing element, an original model trained using a training data set without using a loss function can be retrained to prune or reduce large weights while minimizing the difference between the output of the original model and the output of the retrained model in the retraining operation. Therefore, the retrained model can be accelerated in the computing device with less energy consumption than the original model.

[0032] For example, when an accelerator in a computing device is configured with a memristor as a computing element, an original model trained using a training data set without using a loss function can be retrained to increase the content of 1 bits (or 0 bits) while minimizing the difference between the output of the original model and the output of the retrained model in the retraining operation. Therefore, the retrained model can be accelerated in the computing device with less energy consumption than the original model.

[0033] Figure 1 Customization of training of a weight matrix of an artificial neural network based on characteristics of an accelerator used to accelerate computation of the artificial neural network according to one embodiment is described.

[0034] exist Figure 1 In the training data set 101, a training data set 101 can be used to train a model of an artificial neural network. The artificial neural network can have adjustable parameters configured in the form of a weight matrix. A set of inputs to several artificial neurons can be combined via the weight matrix to generate weighted inputs to the corresponding artificial neurons. Each artificial neuron can generate an output in response to the combined input and the activation function; and the output of the artificial neuron can be connected as an input to another artificial neuron.

[0035] The techniques of machine learning 103 may be used to adjust the weight matrix of the artificial neural network to produce outputs in a manner similar to the training data set 101. For example, the training data set 101 may include inputs and outputs in response to the inputs. The machine learning 103 may adjust the weight matrix of the artificial neural network to minimize the difference between the outputs specified for the inputs in the training data set 101 and the corresponding outputs produced by the artificial neural network with the trained weight matrix.

[0036] exist Figure 1In the present invention, the technique of machine learning 103 is further enhanced with a loss function (e.g., 117 or 127) to customize the trained weight matrix (e.g., 105 or 107) for an accelerator (e.g., 111 or 121) used to accelerate multiplication and accumulation calculations based on the weight matrix (e.g., 105 or 107) to improve energy efficiency.

[0037] For example, accelerator 111 has a computational element of type 113. Therefore, accelerator 111 has a characteristic 115 that indicates an energy consumption pattern when performing computations using weight matrix 105. Loss function 117 in the back-propagation phase of machine learning 103 can be configured according to characteristic 115 to suppress one or more weight patterns in weight matrix 105 and support one or more alternative weight patterns, so that computations performed by accelerator 111 according to trained weight matrix 105 are reduced or minimized.

[0038] For example, machine learning 103 adjusts weight matrix 105 to not only reduce or minimize the difference between the output generated via weight matrix 105 and the output in training data set 101 , but also reduce or minimize loss function 117 .

[0039] Similarly, accelerator 121 has a computing element of type 123. Therefore, accelerator 121 has a characteristic 125 indicating an energy consumption pattern when performing calculations using weight matrix 107. Loss function 127 in the back-propagation phase of machine learning 103 can be configured according to characteristic 125 to suppress one or more weight patterns in weight matrix 107 and support one or more alternative weight patterns, so that the calculations performed by accelerator 121 according to trained weight matrix 107 are reduced or minimized. Machine learning 103 adjusts weight matrix 107 to reduce or minimize not only the difference between the output generated via weight matrix 107 and the output in training data set 101, but also to reduce or minimize loss function 127.

[0040] For example, the energy consumption profile of accelerator 111 may consume more energy for operations on smaller magnitude weights than for operations on larger magnitude weights (e.g., as in an accelerator having a microring resonator as a computational element). Therefore, loss function 117 may be constructed to penalize small weights in weight matrix 105 to promote large weights.

[0041] For example, the energy consumption pattern of the accelerator 121 may consume more energy for operations on larger magnitude weights than for operations on smaller magnitude weights (e.g., as in an accelerator having memristors as computational elements). Therefore, the loss function 127 may be constructed to penalize large weights in the weight matrix 107 and thus promote small weights.

[0042] For example, the energy consumption profile of the accelerator 111 may consume more energy for operations on larger magnitude weights than for operations on smaller magnitude weights (e.g., as in an accelerator having memristors as computational elements). Therefore, the loss function 117 may be constructed to penalize large weights in the weight matrix 105 and thus promote small weights.

[0043] For example, the energy consumption pattern of the accelerator 111 may consume more energy for operations on weights with more bits having a value of 1 (1-bit) than weights with fewer 1-bits (e.g., as in an accelerator having synaptic memory cells as computational elements). Therefore, the loss function 117 may be constructed to penalize 1-bits and thus promote 0-bits in the weight matrix 105. In some cases where the weight matrix 105 has more 0-bits (e.g., weight bits having a value of 0) than 1-bits, the loss function 117 may be configured to penalize 1-bits and thus promote 0-bits; and the accelerator 111 may be used to operate on a bit-wise inverted version of the weight matrix 105 when performing computations for the artificial neural network.

[0044] exist Figure 1 In the embodiment, the artificial neural network is trained using machine learning 103 based not only on the training data set 101 of the specified input and output relationship samples but also on the loss function (e.g., 117, 127). The loss function (e.g., 117, 127) is configured to represent the selection of the weight pattern according to the energy usage characteristics (e.g., 115, 125) of the accelerator (e.g., 111, 121). Therefore, when the accelerator (e.g., 111, 121) is used to accelerate the multiplication and accumulation calculations involved in the use of the weight matrix (e.g., 105, 107) of the artificial neural network, the energy consumption of the calculation is reduced.

[0045] When the energy consumption of the accelerator is substantially independent of the weight patterns in the weight matrix (e.g., an accelerator having logic circuits as computing elements), machine learning 103 can be applied without using this loss function (e.g., 117 or 127) that is configured to fine-tune the weight patterns in the trained weight matrix (e.g., 105 or 107).

[0046] In general, the use of such a loss function (e.g., 117 or 127) may reduce the accuracy of the trained weight matrix (e.g., 105 or 107). However, the reduced energy consumption may be beneficial at the expense of a limited reduction in accuracy. The use of the loss function 117 may be configured to balance the reduction in energy consumption with the reduction in accuracy. For example, a cost weight may be applied to the output of the loss function 127 and the difference between the output generated via the trained weight matrix (e.g., 105, 107) and the output in the training data set 101 to assess the cost of the combination. The cost weight may be adjusted to balance the accuracy target against the energy reduction target.

[0047] Optionally, as in Figure 2 In the present invention, a weight matrix obtained without using a loss function (e.g., 117 or 127) (e.g., suitable for an accelerator having logic circuits as computing elements) is pruned, customized, and retrained to produce a customized weight matrix for an accelerator having a preference for a weight pattern that reduces energy consumption.

[0048] Figure 2 Retraining a weight matrix of an artificial neural network to improve energy efficiency in accelerating computation of the artificial neural network according to one embodiment is described.

[0049] exist Figure 2 In the embodiment of the present invention, the weight matrix 106 can be customized for accelerators (e.g., 111, 121) with different energy usage characteristics (e.g., 115 and 125) (e.g., trained for accelerators having logic circuits as computing elements). The energy usage characteristics (e.g., 115 and 125) can be used to prune or adjust the weight matrix 106; and retraining 104 can be used to minimize the output difference between the original weight matrix 106 and the customized weight matrix (e.g., 105 and 107).

[0050] For example, a pruning selection (e.g., 119 or 129) may be used to identify a set of weights in the weight matrix 106 and modify the selected weights to fine-tune the weight pattern in the weight matrix 106. Retraining 104 may adjust the remaining weights to best match the output of the original weight matrix 106 with the output of the retrained weight matrix (e.g., 105 or 107). Optionally, the selections and modifications in the pruning selection (e.g., 119 or 129) may be adjusted to balance the combined cost goals of accuracy and energy reduction when generating the retrained weight matrix (e.g., 105 or 107).

[0051] For example, after retraining 104, the energy performance of the trained weight matrix (e.g., 105 or 107) may be evaluated. In addition, the accuracy performance of the trained weight matrix (e.g., 105 or 107) is also evaluated. The combined performance metric may be a weighted average of the energy performance and the accuracy performance. The weight selection and modification operations in the pruning selection (e.g., 119 or 129) may be adjusted to search for selection and modification solutions that improve or optimize the combined performance metric.

[0052] For example, the accelerator 121 may have the characteristic of consuming more energy for operations on smaller magnitude weights than for operations on larger magnitude weights (e.g., as in an accelerator having a microring resonator as a computational element). To generate a customized weight matrix 107 for the accelerator 121, the pruning selection 129 may be configured to remove or add small magnitude weights to promote large weights in the customized weight matrix 107 with a limited reduction in accuracy.

[0053] For example, the accelerator 121 may have the property of consuming more energy for operations on larger magnitude weights than for operations on smaller magnitude weights (e.g., as in an accelerator having memristors as computational elements). To generate a customized weight matrix 107 for the accelerator 121, the pruning selection 129 may be configured to remove or reduce large magnitude weights to promote small weights in the customized weight matrix 107 with a limited reduction in accuracy.

[0054] For example, the accelerator 121 may have the characteristic of consuming more energy for operations on weights with more 1 bits than for operations on weights with fewer 1 bits (e.g., as in an accelerator having memristors as computational elements). To generate a customized weight matrix 107 for the accelerator 121, the pruning selection 129 may be configured to selectively invert 1 bits of the weight matrix 106 to promote 0 bits in the customized weight matrix 107 with a limited reduction in accuracy.

[0055] Figure 3 The energy consumption characteristics of some types of accelerators for customized training of artificial neural networks according to some embodiments are shown. The characteristics can be used to customize Figure 1 and Figure 2 Training or retraining of deep learning models in .

[0056] The digital accelerator 131 may be implemented using a logical multiplication-accumulation unit 141. For example, the digital accelerator 131 may have a Fig. 9 The matrix-matrix unit 321 configured in Fig.10 The matrix-vector unit 341 configured in Fig.11 The vector-vector unit 361 configured in the embodiment and the multiplication-accumulation units 371, ..., 373 implemented using logic circuits. This digital accelerator 131 may have an energy consumption characteristic 151 without a weight preference 152. Therefore, the training data set 101 may be trained via machine learning 103 without using a loss function to generate an original weight matrix 106 with a high level of accuracy.

[0057] The photon accelerator 133 may be implemented using a microring resonator 143, such as Figure 4 and Figure 5. This photon accelerator 133 may have an energy consumption characteristic 153 that consumes more energy 154 for small weights. Therefore, the training data set 101 may be trained via machine learning 103 using a loss function (e.g., 117 or 127) configured according to the characteristic 153 to suppress small weights and promote large weights. Alternatively, the original weight matrix 106 with a high level of accuracy may be retrained 104 using a pruning selection (e.g., 119 or 129) configured according to the characteristic 153 to suppress small weights and promote large weights.

[0058] The analog computing module 135 may use the synaptic memory cell array 145 to accelerate multiplication and accumulation operations, such as Figure 6 , Figure 7 and Figure 8 . This analog computing module 135 may have an energy consumption characteristic 155 that consumes more energy 156 for non-0 bits. Therefore, the training data set 101 may be trained via machine learning 103 to suppress non-0 bits (or suppress 0 bits and use an inverted matrix in the calculation) using a loss function (e.g., 117 or 127) configured according to the characteristic 155. Alternatively, the original weight matrix 106 with a high level of accuracy may be retrained 104 using a pruning selection (e.g., 119 or 129) configured according to the characteristic 155 to increase the content of 0 bits (or 1 bits) in the retrained weight matrix (e.g., 105 or 107).

[0059] The electric accelerator 137 may use the memristor 147 to perform multiplication operations. This electric accelerator 137 may have an energy consumption characteristic 157 that consumes more energy 158 for large weights. Therefore, the training data set 101 may be trained via machine learning 103 using a loss function (e.g., 117 or 127) configured according to the characteristic 157 to suppress large weights and promote small weights. Alternatively, the original weight matrix 106 with a high level of accuracy may be retrained 104 using a pruning selection (e.g., 119 or 129) configured according to the characteristic 157 to suppress large weights and promote small weights.

[0060] Figure 4 An analog accelerator implemented using a microring resonator according to one embodiment is shown. For example, Figure 3 The photon accelerator 133 can be Figure 4 implemented in the manner described in the previous section.

[0061] exist Figure 4 In the embodiment of the present invention, the digital-to-analog converter 179 may convert a digital input (eg, the weight matrix 106 , 105 , or 107 ) into a corresponding analog input 170 ; and the analog output 180 may be converted into a digital form via the analog-to-digital converter 189 .

[0062] Figure 4The analog accelerator has microring resonators 181, 182, ..., 183 and 184, and a light source 190 (e.g., a semiconductor laser diode such as a vertical cavity surface emitting laser (VCSEL)) configured to feed light input into waveguides 191, ..., 192.

[0063] Each of the waveguides (eg, 191 or 192) is configured with a plurality of microring resonators (eg, 181, 182; or 183, 184) to change the magnitude of light passing through the corresponding waveguide (eg, 191 or 192).

[0064] A tuning circuit (eg, 171 , 172 , 173 , or 174 ) of a microring resonator (eg, 181 , 182 , 183 , or 184 ) may change the resonant characteristics of the microring resonator (eg, 181 , 182 , 183 , or 184 ) by heat or carrier injection.

[0065] Thus, the ratio between the magnitude of light exiting a waveguide (e.g., 191) and entering the combined waveguide 194 and the magnitude of light entering a waveguide (e.g., 191) near the light source 190 represents a multiplication of the attenuation factor implemented by the tuning circuits (e.g., 171 and 172) of the microring resonators (e.g., 181 and 182) that electromagnetically interact with the waveguide (e.g., 191).

[0066] The combining waveguide 194 sums the results of the multiplications performed via the light passing through the waveguides 191, ..., 192. The photodetector 193 is configured to convert the combined light output from the waveguides into an analog output 180 in the electrical domain.

[0067] For example, a set of inputs from an input weight matrix (e.g., 106, 105, or 107) may be applied to tuning circuits 171, ..., 173 as part of analog input 170; and a set of weight elements from a row of the weight matrix (e.g., 106, 105, or 107) may be applied to tuning circuits 172, ..., 174 via another portion of analog input 170; and the output of combining waveguide 194 to photodetector 193 represents multiplication and accumulation between a set of inputs weighted via a set of weight elements. Analog-to-digital converter 189 may convert analog output 180 into an output.

[0068] The same set of input elements as applied via tuning circuits 171, ..., 173 may be maintained while a set of weight elements from a next row of a weight matrix (e.g., 106, 105, or 107) may be applied to tuning circuits 172, ..., 174 via a portion of analog input 170 to perform multiplication and accumulation of the next row of weights on the input elements. After the calculations involving the same set of input elements are completed, the next set of input elements may be loaded from the input matrix.

[0069] Alternatively, the same set of weight elements from a row of a weight matrix (e.g., 106, 105, or 107) may be maintained for different sets of input elements (e.g., via tuning circuits 172, ..., 174 to a portion of analog input 170). After completing calculations involving the same set of weight elements, the next set of weight elements may be loaded from the weight matrix.

[0070] Alternatively, the inputs may be applied via tuning circuits 172 , . . . , 174 ; and the weight elements may be applied via tuning circuits 171 , . . . , 173 .

[0071] Figure 5 Another accelerator implemented using a microring resonator according to one embodiment is shown. For example, Figure 3 The photon accelerator 133 can be Figure 5 implemented in the manner described in the previous section.

[0072] and Figure 4 Similar to the simulation accelerator, Figure 5 The analog accelerator has microring resonators 181, 182, ..., 183 and 184, tuning circuits 171, 172, ..., 173 and 174, waveguides 191, ..., and 192, and a combined waveguide 194.

[0073] exist Figure 5 In FIG. 1 , the analog accelerator has amplitude controls 161, ..., and 163 for light sources 162, ..., 164 connected to waveguides 191, ..., and 192, respectively. Thus, the amplitude of light entering the waveguides 191, ..., and 192 can be controlled via a portion of the analog input 170 connected to the amplitude controls 161, ..., 163. The amplitude of light exiting the waveguide (e.g., 191) represents the product of the input of the amplitude control (e.g., 161) to the light source (e.g., 162) of the waveguide (e.g., 191) and the input to the tuning circuit (e.g., 171 and 172) of the microring resonator (e.g., 181 and 182) interacting with the waveguide (e.g., 191).

[0074] For example, inputs from an input weight matrix (e.g., 106, 105, or 107) may be applied via amplitude controls 161, ..., 163; weight elements from a weight matrix (e.g., 106, 105, or 107) may be applied via tuning circuits 171, ..., 173 (or 172, ..., 174); and optional scaling factors may also be applied via tuning circuits 172, ..., 174 (or 171, ..., 173).

[0075] Alternatively, inputs from an input weight matrix (e.g., 106, 105, or 107) may be applied via tuning circuits 171, ..., 173 (or 172, ..., 174); and weight elements from a weight matrix (e.g., 106, 105, or 107) may be applied via amplitude controls 161, ..., 163.

[0076] Optionally, the microring resonators 182, ..., 184 and their tuning circuits 172, ..., 174 may be omitted. The scaling factor may be applied by the accelerator manager.

[0077] Figure 6 The calculation of multiplying the weight bit column by the input bit column to provide an accumulated result is shown according to one embodiment. For example, Figure 3 The synaptic memory cell array 145 in the analog computing module 135 can be as follows Figure 6 The method described in is configured to perform multiplication and accumulation operations.

[0078] exist Figure 6 , columns of synaptic memory cells 207, 217, ..., 227 (e.g., in the memory cell array 145 of the analog computing module 135) can be programmed in synaptic mode to have a threshold voltage at a level representing a weight of storing one bit per memory cell.

[0079] Columns of memory cells 207, 217, ..., 227 programmed in synaptic mode can be read in synaptic mode, during which voltage drivers 203, 213, ..., 223 are configured to concurrently apply voltages 205, 215, ..., 225 to memory cells 207, 217, ..., 227, respectively, based on their received input bits 201, 211, ..., 221.

[0080] For example, when the input bit 201 has a value of 1, the voltage driver 203 applies a predetermined read voltage as the voltage 205, thereby causing the memory cell 207 to output a predetermined amount of current as its output current 209 (provided that the memory cell 207 has a threshold voltage programmed at a lower level, which is lower than the predetermined read voltage) to represent a stored weight of 1, or output a negligible amount of current as its output current 209 (provided that the memory cell 207 has a threshold voltage programmed at a higher level, which is higher than the predetermined read voltage) to represent a stored weight of 0. However, when the input bit 201 has a value of 0, the voltage driver 203 applies a voltage (e.g., 0) lower than the lower level of the threshold voltage as the voltage 205 (e.g., no predetermined read voltage is applied), thereby causing the memory cell 207 to output a negligible amount of current as its output current 209, regardless of the weight stored in the memory cell 207. Therefore, the output current 209, which is a multiple of the predetermined amount of current, represents the result of multiplying the weight bit stored in the memory cell 207 by the input bit 201.

[0081] Similarly, current 219 passing through memory cell 217 as a multiple of a predetermined amount of current represents the result of multiplying the weight bits stored in memory cell 217 by input bit 211; and current 229 passing through memory cell 227 as a multiple of a predetermined amount of current represents the result of multiplying the weight bits stored in memory cell 227 by input bit 221.

[0082] The output currents 209, 219, ..., and 229 of the memory cells 207, 217, ..., 227 are connected to a common line 241 (e.g., a bit line) for summing. The summed current 231 is compared with a unit current 232 equal to a predetermined amount of current by a digitizer 233 of an analog-to-digital converter 245 to determine a digital result 237 of multiplying the weight bit columns stored in the memory cells 207, 217, ..., 227, respectively, by the columns of input bits 201, 211, ..., 221, respectively, and the sum of the multiplication results.

[0083] The sum of the negligible amounts of current from the memory cells connected to line 241 is smaller when compared to the unit current 232 (e.g., the predetermined amount of current). Therefore, the presence of the negligible amounts of current from the memory cells does not change the result 237 and is negligible in the operation of the analog-to-digital converter 245.

[0084] exist Figure 6, the voltages 205, 215, ..., 225 applied to the memory cells 207, 217, ..., 227 represent the digitized input bits 201, 211, ..., 221; the memory cells 207, 217, ..., 227 are programmed to store digitized weight bits; and the currents 209, 219, ..., 229 represent the digitized results. Therefore, the memory cells 207, 217, ..., 227 do not act as memristors that convert analog voltages into analog currents based on their linear resistance over a voltage range; and the operating principles of the memory cells when calculating multiplications are fundamentally different from the operating principles of a memristor crossbar switch. When a memristor crossbar switch is used, a conventional digital-to-analog converter is used to generate an input voltage proportional to the input to be applied to the memristor crossbar switch row. When using Figure 6 When the technology is used, such a digital-to-analog converter can be eliminated; and the operation of the digitizer 233 to produce the result 237 can be greatly simplified. The result 237 is an integer not greater than the count of the memory cells 207, 217, ..., 227 connected to the line 241. The digitized form of the output current 209, 219, ..., 229 can improve the accuracy and reliability of the calculations performed using the memory cells 207, 217, ..., 227.

[0085] In general, the weights involved in multiplication and accumulation operations can be more than one bit. Figure 6 As described in , multiple memory cell columns can be used to store different significant bits of weights to perform multiplication and accumulation operations.

[0086] Figure 6 The circuit described in can be considered as a multiplier-accumulator unit configured to operate on a 1-bit weight column and a 1-bit input column. Multiple such circuits can be connected in parallel to implement a multiplier-accumulator unit to operate on a multi-bit weight column and a 1-bit input column, such as Figure 6 As described in .

[0087] Figure 6The illustrated circuit can also be used to read data stored in memory cells 207, 217, ..., 227. For example, to read data or weights stored in memory cell 207, input bits 211, ..., 221 can be set to 0 to cause memory cells 217, ..., 227 to output a negligible amount of current into line 241 (e.g., as a bit line). Input bit 201 is set to 1 to cause voltage driver 203 to apply a predetermined read voltage. Thus, the result 237 from digitizer 233 provides the data or weights stored in memory cell 207. Similarly, data or weights stored in memory cell 217 can be read by applying 0 as input bit 211 and applying 0 as the remaining input bits in the column; and data or weights stored in memory cell 227 can be read by applying 1 as input bit 221 and applying 0 as the other input bits in the column.

[0088] Generally speaking, Figure 6 The circuits described in can be used to select any of the memory cells 207, 217, ..., 227 for reading or writing. A voltage driver (e.g., 203) can apply a programming voltage pulse to adjust the threshold voltage of the corresponding memory cell (e.g., 207) to erase data, store data or weight, etc.

[0089] Figure 7 A calculation is shown that multiplies a multi-bit weight column by an input bit column to provide an accumulated result according to one embodiment.

[0090] exist Figure 7 , the weight 250 in binary form has a most significant bit 257, a second most significant bit 258, ..., and a least significant bit 259. The significant bits 257, 258, ..., 259 may be stored in rows of memory cells 207, 206, ..., 208 across several columns in the array 273, respectively (e.g., in the memory cell array 145 of the analog computing module 135). The significant bits 257, 258, ..., 259 of the weight 250 are to be multiplied by the input bit 201 (e.g., as represented by the voltage 205 applied by the voltage driver 203 on the line 281 (e.g., the word line) Figure 6 ).

[0091] Similarly, memory cells 217, 216, ..., 218 may be used to store corresponding valid bits (e.g., as shown in FIG. 1 ) of the next weight to be multiplied by the next input bit 211 represented by the voltage 215 applied by the voltage driver 213 on the line 282 (e.g., the word line). Figure 6 and memory cells 227, 226, ..., 228 may be used to store corresponding weights (eg, as Figure 6).

[0092] like Figure 6 As shown in , the most significant bit (e.g., 257) of the weight (e.g., 250) stored in the corresponding row of memory cells in array 273 is multiplied by the input bits 201, 211, ..., 221 represented by voltages 205, 215, ..., 225 and then summed into a current 231 in line 241 and digitized using a digitizer 233 to produce a result 237 corresponding to the most significant bit of the weight.

[0093] Similarly, the next most significant bit (e.g., 258) of the weight (e.g., 250) stored in the corresponding row of memory cells in array 273 is multiplied by the input bits 201, 211, ..., 221 represented by voltages 205, 215, ..., 225 and then summed into a current in line 242 and digitized to produce a result 236 corresponding to the next most significant bit.

[0094] Similarly, the least significant bit (e.g., 259) of the weight (e.g., 250) stored in the corresponding row of memory cells in array 273 is multiplied by the input bits 201, 211, ..., 221 represented by voltages 205, 215, ..., 225 and then summed into a current in line 243 and digitized to produce a result 238 corresponding to the least significant bit.

[0095] The most significant bit may be shifted left by one bit to have the same weight as the second significant bit, which may be further shifted left by one bit to have the same weight as the next significant bit. Thus, the result 237 generated from the multiplication and summation of the most significant bit (e.g., 257) of the weight (e.g., 250) may be applied with the one-bit-left-shift operation 247; and the addition operation 246 may be applied to the result of the left shift operation 247 and the result 236 generated from the multiplication and summation of the second most significant bit (e.g., 258) of the weight (e.g., 250). The left shift operation (e.g., 247, 249) may be used to apply the weights of the bits (e.g., 257, 258, ...) to sum using the addition operation (e.g., 246, ..., 248) to generate the result 251. Thus, the result 251 is equal to the weight column in the memory cell array 273 multiplied by the column of input bits 201, 211, ..., 221 and accumulating the multiplication results.

[0096] In general, the inputs involved in the multiplication and accumulation operations may be more than one bit. Multiple input bit columns may be applied one column at a time to the weights stored in the memory cell array 273 to obtain the result of multiplying the weight column by the input column and accumulating the results, such as Figure 8 As described in .

[0097] Figure 7The circuits described in can be used to read data stored in the memory cell array 273. For example, to read the data or weights 250 stored in the memory cells 207, 206, ..., 208, the input bits 211, ..., 221 can be set to 0 to cause the memory cells 217, 216, ..., 218, ..., 227, 226, ..., 228 to output a negligible amount of current into the lines 241, 242, ..., 243 (e.g., as bit lines). The input bit 201 is set to 1 to cause the voltage driver 203 to apply a predetermined read voltage as the voltage 205. Thus, the results 237, 236, ..., 238 from the digitizer (e.g., 233) connected to the lines 241, 242, ..., 243 provide the bits 257, 258, ..., 259 of the data or weights 250 stored in the row of the memory cells 207, 206, ..., 208. Furthermore, the result 251 calculated from the shift operations 247, ..., 249 and the addition operations 246, ..., 248 provides a weight 250 in binary form.

[0098] Generally speaking, Figure 7 The circuits described in can be used to select any row of the memory cell array 273 for reading. Optionally, different columns of the memory cell array 273 can be driven by different voltage drivers. Thus, the memory cells in a row (e.g., 207, 206, ..., 208) can be programmed to write the data of the weight 250 in parallel (e.g., to store bits 257, 258, ..., 259).

[0099] Figure 8 A calculation is shown that multiplies a multi-bit weight column by a multi-bit input column to provide an accumulated result according to one embodiment.

[0100] exist Figure 8 , valid bits of an input (eg, 280) are applied to a multiplier-accumulator unit 270 at a plurality of times T, T1, . . . , T2.

[0101] For example, multi-bit input 280 may have most significant bit 201 , next most significant bit 202 , . . . , least significant bit 204 .

[0102] At time T, the most significant bits 201, 211, ..., 221 of the input (e.g., 280) are applied to the multiplier-accumulator unit 270 to obtain the result 251 of multiplying the weight (e.g., 250) stored in the memory cell array 273 by the column of bits 201, 211, ..., 221 and the sum of the multiplication results.

[0103] For example, the multiplier-accumulator unit 270 may be configured as follows: Figure 7The multiplier-accumulator unit 270 has a voltage driver 271 connected to apply voltages 205, 215, ..., 225 representing input bits 201, 211, ..., 221. The multiplier-accumulator unit 270 has a memory cell array 273 storing weight bits, such as Figure 7 The multiplier-accumulator unit 270 has a digitizer 275 for converting the current summed on the lines 241, 242, ..., 243 of the memory cell columns in the array 273 into output results 237, 236, ..., 238. The multiplier-accumulator unit 270 has a shifter 277 and an adder 279 connected to combine the column results 237, 236, ..., 238 to provide a result 251, as shown in FIG. Figure 7 In some implementations, the logic circuitry of the multiplier-accumulator unit 270 (eg, the shifter 277 and the adder 279 ) is implemented as part of the inference logic circuitry of the analog computation module 135 .

[0104] Similarly, at time T1, the next most significant bits 202, 212, ..., 222 of the input (e.g., 280) are applied to the multiplier-accumulator unit 270 to obtain the result 253 of multiplying the weights (e.g., 250) stored in the memory cell array 273 by the vector of bits 202, 212, ..., 222 and the sum of the multiplication results.

[0105] Similarly, at time T2, the least significant bits 204, 214, ..., 224 of the input (e.g., 280) are applied to the multiplier-accumulator unit 270 to obtain the result 255 of the vector of bits 202, 212, ..., 222 stored in the memory cell array 273 multiplied by the weights (e.g., 250) and the sum of the multiplication results.

[0106] The result 251 generated from the multiplication and summation of the most significant bits 201, 211, ..., 221 of the input (e.g., 280) may be applied with a one-bit left shift operation 261; and an addition operation 262 may be applied to the result of the left shift operation 261 and the result 253 generated from the multiplication and summation of the second most significant bits 202, 212, ..., 222 of the input (e.g., 280). The left shift operation (e.g., 261, 263) may be used to apply the weights of the bits (e.g., 201, 202, ...) to be summed using addition operations (e.g., 262, ..., 264) to generate a result 267. Therefore, the result 267 is equal to the weights (e.g., 250) in the memory cell array 273 multiplied by the input columns (e.g., 280) respectively and then summed.

[0107] A plurality of multiplier-accumulator units 270 may be connected in parallel to operate on a weight matrix that multiplies a multi-bit input column at a series of times T, T1, ..., T2.

[0108] Figure 3 The simulation calculation module 135 can be configured as follows Figure 6 , Figure 7 and Figure 8 The multiplication and accumulation operations are performed in the manner described in .

[0109] Fig. 9 A processing unit 321 is shown configured to perform matrix-matrix operations according to one embodiment. For example, Figure 3 The logical multiplication-accumulation unit 141 of the digital accelerator 131 may be configured as Fig. 9 Matrix-matrix unit 321.

[0110] exist Fig. 9 , the matrix-matrix unit 321 includes a plurality of kernel buffers 331 to 333 and a plurality of mapping banks 351 to 353. Each of the mapping banks 351 to 353 stores one vector of matrix operands having a plurality of vectors respectively stored in the mapping banks 351 to 353; and each of the kernel buffers 331 to 333 stores one vector of another matrix operand having a plurality of vectors respectively stored in the kernel buffers 331 to 333. The matrix-matrix unit 321 is configured to perform multiplication and accumulation operations on elements of two matrix operands using a plurality of matrix-vector units 341 to 343 operating in parallel.

[0111] The crossbar switch 323 connects the mapping memory banks 351 to 353 to the matrix-vector units 341 to 343. The same matrix operand stored in the mapping memory banks 351 to 353 is provided to each of the matrix-vector units 341 to 343 via the crossbar switch 323; and the matrix-vector units 341 to 343 receive data elements from the mapping memory banks 351 to 353 in parallel. Each of the kernel buffers 331 to 333 is connected to a corresponding one of the matrix-vector units 341 to 343 and provides a vector operand to the corresponding matrix-vector unit. The matrix-vector units 341 to 343 operate concurrently to calculate the operation of multiplying the same matrix operand stored in the mapping memory banks 351 to 353 by the corresponding vector stored in the kernel buffers 331 to 333. For example, the matrix-vector unit 341 performs multiplication operations on matrix operands stored in the mapping memory 351 to 353 and vector operands stored in the kernel buffer 331, while the matrix-vector unit 343 concurrently performs multiplication operations on matrix operands stored in the mapping memory 351 to 353 and vector operands stored in the kernel buffer 333.

[0112] Fig. 9 Each of the matrix-vector units 341 to 343 in can be as follows Fig.10Implemented in the manner described in .

[0113] Fig.10 A processing unit 341 is shown configured to perform matrix-vector operations according to one embodiment. For example, Fig.10 The matrix-vector unit 341 can be used as Fig. 9 Any one of the matrix-vector units in the matrix-matrix unit 321 of .

[0114] exist Fig.10 In the example, each of the memory banks 351 to 353 is mapped to Fig. 9 The mapping memory banks 351 to 353 store one vector of matrix operands having a plurality of vectors respectively stored in the mapping memory banks 351 to 353 in a similar manner. Fig.10 The crossbar switch 323 in the kernel buffer 331 supplies the vectors from the mapping memory bank 351 to the vector-vector units 361 to 363, respectively. The same vectors stored in the kernel buffer 331 are supplied to the vector-vector units 361 to 363.

[0115] The vector-vector units 361 to 363 operate concurrently to calculate operations of multiplying corresponding vector operands stored in the mapped memory banks 351 to 353, respectively, by the same vector operands stored in the kernel buffer 331. For example, the vector-vector unit 361 performs a multiplication operation on a vector operand stored in the mapped memory bank 351 and a vector operand stored in the kernel buffer 331, while the vector-vector unit 363 concurrently performs a multiplication operation on a vector operand stored in the mapped memory bank 353 and a vector operand stored in the kernel buffer 331.

[0116] when Fig.10 The matrix-vector unit 341 in Fig. 9 When implemented in the matrix-matrix unit 321 of FIG. 3 , the matrix-vector unit 341 may use the mapping memory banks 351 to 353 , the crossbar switch 323 , and the kernel buffer 331 of the matrix-matrix unit 321 .

[0117] Fig.10 Each of the vector-vector units 361 to 363 in Fig.11 Implemented in the manner described in .

[0118] Fig.11 A processing unit 361 is shown configured to perform vector-vector operations according to one embodiment. For example, Fig.11 The vector-vector unit 361 can be used as Fig.10 Any one of the vector-vector units in the matrix-vector unit 341 of .

[0119] exist Fig.11, the vector-vector unit 361 has a plurality of multiply-accumulate (MAC) units 371 to 373. Each of the multiply-accumulate (MAC) units 371 to 373 may receive two numbers as operands, perform a multiplication operation of the two numbers, and add the multiplication result to the sum maintained in the multiply-accumulate (MAC) unit.

[0120] Each of the vector buffers 381 and 383 stores a list of numbers. A pair of numbers, each from one of the vector buffers 381 and 383, may be provided as input to each of the multiplication-accumulation (MAC) units 371 to 373. The multiplication-accumulation (MAC) units 371 to 373 may receive multiple pairs of numbers from the vector buffers 381 and 383 in parallel and perform multiplication-accumulation (MAC) operations in parallel. The outputs from the multiplication-accumulation (MAC) units 371 to 373 are stored in the shift register 375; and the accumulator 377 calculates the sum of the results in the shift register 375.

[0121] when Fig.11 The vector-vector unit 361 in Fig.10 When implemented in the matrix-vector unit 341 of the matrix-vector unit 341, the vector-vector unit 361 may use the mapped memory bank (e.g., 351 or 353) as one vector buffer 381 and use the kernel buffer 331 of the matrix-vector unit 341 as another vector buffer 383.

[0122] The vector buffers 381 and 383 may have the same length to store the same number / count of data elements. The length may be equal to or a multiple of the count of the multiplication-accumulation (MAC) units 371 to 373 in the vector-vector unit 361. When the length of the vector buffers 381 and 383 is a multiple of the count of the multiplication-accumulation (MAC) units 371 to 373, in each iteration, a number of pairs of inputs equal to the count of the multiplication-accumulation (MAC) units 371 to 373 may be provided as inputs from the vector buffers 381 and 383 to the multiplication-accumulation (MAC) units 371 to 373; and the vector buffers 381 and 383 feed their elements into the multiplication-accumulation (MAC) units 371 to 373 through multiple iterations.

[0123] In one embodiment, the communication bandwidth of the bus between the digital accelerator 131 and the memory is sufficient to allow the matrix-matrix unit 321 to use portions of the memory as mapping banks 351 - 353 and kernel buffers 331 - 333 .

[0124] In another embodiment, the mapped memory banks 351-353 and the kernel buffers 331-333 are implemented in a portion of the local memory of the digital accelerator 131. The communication bandwidth of the bus 111 between the digital accelerator 131 and the memory is sufficient to load the matrix operands for the next operation cycle of the matrix-matrix unit 321 into another portion of the local memory while the matrix-matrix unit 321 performs calculations in the current operation cycle using the mapped memory banks 351-353 and the kernel buffers 331-333 implemented in a different portion of the local memory of the digital accelerator 131.

[0125] Fig.12 An example computing system with an accelerator is shown according to one embodiment.

[0126] Fig.12 The example computing system of includes a host system 410 and a memory subsystem 401. The accelerator 100 may be configured in the memory subsystem 401 or in the host system 410, or in both. The accelerator 100 may include a digital accelerator 131, a photon accelerator 133, an analog computing module 135, an electric accelerator 137, or another type of accelerator. In some embodiments, a portion of the accelerator 100 is implemented in the memory subsystem 401, and another portion of the accelerator 100 is implemented in the host system 410.

[0127] For example, Figure 1 Machine Learning 103 or Figure 2 Retraining 104 is available at Fig.12 The accelerator 100 may be used to accelerate multiplication and accumulation operations performed during machine learning 103 or retraining 104.

[0128] For example, deep learning models can use Figure 1 or Figure 2 technology to customize for Fig.12 For example, Figure 1 Machine Learning 103 or Figure 2 The retraining 104 may be customized based on the energy consumption characteristics of the accelerator 100 to provide for reduced energy consumption with limited accuracy degradation.

[0129] Memory subsystem 401 may include media such as one or more volatile memory devices (eg, memory device 421), one or more non-volatile memory devices (eg, memory device 423), or a combination thereof.

[0130] The memory subsystem 401 may be a storage device, a memory module, or a hybrid of a storage device and a memory module. Examples of storage devices include solid-state drives (SSDs), flash drives, universal serial bus (USB) flash drives, embedded multimedia controller (eMMC) drives, universal flash storage (UFS) drives, secure digital (SD) cards, and hard disk drives (HDDs). Examples of memory modules include dual in-line memory modules (DIMMs), small outline DIMMs (SO-DIMMs), and various types of non-volatile dual in-line memory modules (NVDIMMs).

[0131] The computing system may be a computing device such as a desktop computer, a laptop computer, a network server, a mobile device, a vehicle (e.g., an airplane, drone, train, car, or other transportation vehicle), an Internet of Things (IoT)-enabled device, an embedded computer (e.g., a computer included in a vehicle, industrial equipment, or a networked business device), or such a computing device that includes a memory and a processing device.

[0132] The computing system may include a host system 410 coupled to one or more memory subsystems 401 . Fig.12 An example of a host system 410 coupled to a memory subsystem 401 is illustrated. As used herein, "coupled to" or "coupled with" generally refers to a connection between components, which can be an indirect communication connection or a direct communication connection (e.g., without intervening components), whether wired or wireless, including connections such as electrical connections, optical connections, magnetic connections, etc.

[0133] The host system 410 may include a processor chipset (e.g., processing device 411) and a software stack executed by the processor chipset. The processor chipset may include one or more cores, one or more caches, a memory controller (e.g., controller 413) (e.g., NVDIMM controller) and a storage protocol controller (e.g., PCIe controller, SATA controller). The host system 410 uses the memory subsystem 401, for example, to write data to the memory subsystem 401 and read data from the memory subsystem 401.

[0134] The host system 410 may be coupled to the memory subsystem 401 via a physical host interface 409. Examples of the physical host interface 409 include, but are not limited to, a Serial Advanced Technology Attachment (SATA) interface, a Peripheral Component Interconnect Express (PCIe) interface, a Universal Serial Bus (USB) interface, a Fibre Channel, a Serial Attached SCSI (SAS) interface, a Double Data Rate (DDR) memory bus interface, a Small Computer System Interface (SCSI), a Dual In-line Memory Module (DIMM) interface (e.g., a DIMM slot interface supporting Double Data Rate (DDR)), an Open NAND Flash Interface (ONFI), a Double Data Rate (DDR) interface, a Low Power Double Data Rate (LPDDR) interface, or any other interface. The physical host interface 409 may be used to transfer data between the host system 410 and the memory subsystem 401. When the memory subsystem 401 is coupled to the host system 410 through a PCIe interface, the host system 410 may further utilize an NVM Express (NVMe) interface to access components (e.g., the memory device 423). The physical host interface 409 may provide an interface for passing control, address, data, and other signals between the memory subsystem 401 and the host system 410 . Fig.12 Memory subsystem 401 is illustrated. In general, host system 410 can access multiple memory subsystems via the same communication connection, multiple separate communication connections, and / or a combination of communication connections.

[0135] The processing device 411 of the host system 410 may be, for example, a microprocessor, a central processing unit (CPU), a processing core of a processor, an execution unit, etc. In some examples, the controller 413 may be referred to as a memory controller, a memory management unit, and / or an initiator. In one example, the controller 413 controls communications through a bus coupled between the host system 410 and the memory subsystem 401. In general, the controller 413 may send commands or requests for desired access to the memory devices 423, 421 to the memory subsystem 401. The controller 413 may further include an interface circuit system for communicating with the memory subsystem 401. The interface circuit system may convert responses received from the memory subsystem 401 into information for the host system 410.

[0136] The controller 413 of the host system 410 may communicate with the controller 403 of the memory subsystem 401 to perform operations such as reading data, writing data, or erasing data and other such operations at the memory devices 423, 421. In some examples, the controller 413 is integrated into the same package of the processing device 411. In other examples, the controller 413 is separate from the package of the processing device 411. The controller 413 and / or the processing device 411 may include hardware, such as one or more integrated circuits (ICs) and / or discrete components, buffer memory, cache memory, or a combination thereof. The controller 413 and / or the processing device 411 may be a microcontroller, a dedicated logic circuit system (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), or another suitable processor.

[0137] The memory devices 423, 421 may include any combination of different types of non-volatile memory components and / or volatile memory components. Volatile memory devices (e.g., memory device 421) may be, but are not limited to, random access memory (RAM), such as dynamic random access memory (DRAM) and synchronous dynamic random access memory (SDRAM).

[0138] Some examples of non-volatile memory components include NAND (or NOT AND) type flash memory and write-in-place memory, such as three-dimensional cross-point ("3D cross-point") memory. A cross-point array of non-volatile memory can be combined with a stackable cross-grid data access array to perform bit storage based on changes in bulk resistance. In addition, in contrast to many flash-based memories, cross-point non-volatile memory can perform write-in-place operations, where non-volatile memory cells can be programmed without previously erasing the non-volatile memory cells. NAND type flash memory includes, for example, two-dimensional NAND (2D NAND) and three-dimensional NAND (3D NAND).

[0139] Each of the memory devices 423 may include one or more arrays of memory cells 427. One type of memory cell, such as a single level cell (SLC), may store one bit per cell. Other types of memory cells, such as multi-level cells (MLC), three-level cells (TLC), four-level cells (QLC), and five-level cells (PLC), may store multiple bits per cell. In some embodiments, each of the memory devices 423 may include one or more arrays of memory cells, such as SLC, MLC, TLC, QLC, PLC, or any combination thereof. In some embodiments, a particular memory device may include an SLC portion, an MLC portion, a TLC portion, a QLC portion, and / or a PLC portion of memory cells. The memory cells of the memory devices 423 may be grouped into pages, which may refer to a logical unit of a memory device used to store data. For some types of memory, such as NAND, pages may be grouped to form blocks.

[0140] Although non-volatile memory devices such as 3D cross-point type and NAND type memory (e.g., 2D NAND, 3D NAND) are described, the memory device 423 may be based on any other type of non-volatile memory such as read-only memory (ROM), phase change memory (PCM), self-select memory, other chalcogenide-based memories, ferroelectric transistor random access memory (FeTRAM), ferroelectric random access memory (FeRAM), magnetic random access memory (MRAM), spin transfer torque (STT)-MRAM, conductive bridging RAM (CBRAM), resistive random access memory (RRAM), oxide-based RRAM (OxRAM), NOR flash memory, and electrically erasable programmable read-only memory (EEPROM).

[0141] The memory subsystem controller 403 (or simply controller 403) can communicate with the memory device 423 to perform operations such as reading data, writing data, or erasing data and other such operations at the memory device 423 (e.g., in response to commands dispatched by the controller 413 on the command bus). The controller 403 may include hardware, such as one or more integrated circuits (ICs) and / or discrete components, buffer memory, or a combination thereof. The hardware may include digital circuitry with dedicated (i.e., hard-coded) logic to perform the operations described herein. The controller 403 may be a microcontroller, dedicated logic circuitry (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), or another suitable processor.

[0142] The controller 403 may include a processing device 407 (processor) configured to execute instructions stored in the local memory 405. In the illustrated example, the local memory 405 of the controller 403 includes an embedded memory configured to store instructions for executing various processes, operations, logic flows, and routines that control the operation of the memory subsystem 401, including handling communications between the memory subsystem 401 and the host system 410.

[0143] In some embodiments, local memory 405 may include memory registers that store memory pointers, fetched data, etc. Local memory 405 may also include read-only memory (ROM) for storing microcode. Fig.12 The example memory subsystem 401 in has been illustrated as including a controller 403, but in another embodiment of the present disclosure, the memory subsystem 401 does not include the controller 403, but may rely on external control (e.g., provided by an external host, or by a processor or controller separate from the memory subsystem).

[0144] In general, the controller 403 may receive commands or operations from the host system 410 and convert the commands or operations into instructions or appropriate commands to achieve the desired access to the memory device 423. The controller 403 may be responsible for other operations associated with the memory device 423, such as wear leveling operations, garbage collection operations, error detection and error correction code (ECC) operations, encryption operations, cache operations, and address translation between logical addresses (e.g., logical block addresses (LBA), namespaces) and physical addresses (e.g., physical block addresses). The controller 403 may further include a host interface circuit system to communicate with the host system 410 via a physical host interface. The host interface circuit system may convert commands received from the host system into command instructions to access the memory device 423 and convert responses associated with the memory device 423 into information for the host system 410.

[0145] The memory subsystem 401 may also include additional circuitry or components not illustrated. In some embodiments, the memory subsystem 401 may include a cache or buffer (e.g., DRAM) and address circuitry (e.g., row decoders and column decoders) that may receive addresses from the controller 403 and decode the addresses to access the memory device 423.

[0146] In some embodiments, memory device 423 includes a local media controller 425 that operates in conjunction with memory subsystem controller 403 to perform operations on one or more memory cells of memory device 423. An external controller (e.g., memory subsystem controller 403) can manage memory device 423 externally (e.g., perform media management operations on memory device 423). In some embodiments, memory device 423 is a managed memory device, which is a raw memory device combined with a local controller (e.g., local controller 425) for media management within the same memory device package. An example of a managed memory device is a managed NAND (MNAND) device.

[0147] Fig.13 A method for training a deep learning model according to one embodiment is shown. For example, the method may be Fig.12 For example, a deep learning model can be trained to Fig.12 The invention is executed in a computing system or device.

[0148] At block 501, the type of accelerator for multiply and accumulate operations is identified. A deep learning model may be trained during training (e.g., Figure 1 ) or by retraining (e.g., Figure 2 ) are customized for the type of accelerator.

[0149] For example, a digital accelerator (eg, 131) of one type (eg, 113 or 123) may be implemented using a logical multiply-accumulate unit 141 for multiplication (eg, as Fig. 9 , Fig.10 , Fig.11 ). Therefore, such an accelerator (e.g., 131) may have the characteristic 151 of consuming similar amounts of energy for different weight distribution patterns. It is not necessary to suppress any weight pattern to customize the weight matrix of the artificial neural network in the deep learning model.

[0150] For example, one type of photon accelerator (e.g., 133) may be implemented using a microring resonator 143 for multiplication (e.g., as Figure 4 , Figure 5 ). Therefore, such an accelerator (e.g., 133) may have a characteristic 153 of consuming more energy for weight patterns concentrated in low magnitude regions of the weight value distribution than for weight patterns concentrated in high magnitude regions of the weight value distribution. Therefore, it is preferred to use a loss function (e.g., 117) to suppress weight patterns concentrated in low magnitude regions in the weight value distribution during machine learning 103 based on the training data set 101, or to selectively prune weights via retraining 104 to suppress such patterns.

[0151] For example, one type of electric accelerator (e.g., 137) may be implemented using a memristor 147 for multiplication. Therefore, such an accelerator (e.g., 137) may have a characteristic 157 of consuming more energy for weight patterns concentrated in high magnitude regions of the weight value distribution than for weight patterns concentrated in low magnitude regions of the weight value distribution. Therefore, it is preferred to use a loss function (e.g., 127) to suppress weight patterns concentrated in high magnitude regions in the weight value distribution during machine learning 103 based on the training data set 101, or to selectively prune weights to suppress such patterns via retraining 104.

[0152] For example, one type of analog computing module (e.g., 135) may be implemented using a synaptic memory cell array 145 for multiplication. Therefore, such an accelerator (e.g., analog computing module 135) may have a characteristic 151 of consuming more energy for a weight bit distribution pattern that is more concentrated in bits having a first value (e.g., 1) than bits having a second value (e.g., 0). Therefore, it is preferred to use a loss function (e.g., 127) to suppress the weight bit distribution pattern that is more concentrated in bits having a first value (e.g., 1) during machine learning 103 according to a training data set 101, or selectively prune weight bits having a first value (e.g., by flipping to a second value) to suppress this pattern via retraining 104.

[0153] At block 503 , a weight matrix of the artificial neural network is adjusted based on energy consumption characteristics (eg, 115 or 125 ) of the type (eg, 113 or 123 ) of the accelerator.

[0154] For example, adjusting the weight matrix at block 503 may include training the weight matrix (e.g., 105 or 107) according to the training data set 101 via machine learning 103. The training of the weight matrix (e.g., 105 or 107) may include reducing a loss function (e.g., 117 or 127) associated with the energy consumption characteristic (e.g., 115 or 125).

[0155] For example, to customize the training of a weight matrix (e.g., 105) for an accelerator (e.g., 111) of type 113 implemented using microring resonator 143 as a computational element for multiplication, loss function 117 may be configured to penalize small weights more than large weights.

[0156] For example, to customize the training of a weight matrix (e.g., 107) for an accelerator of type 123 (e.g., 121) implemented using memristors 147 as computational elements for multiplication, loss function 127 may be configured to penalize large weights more than small weights.

[0157] For example, to customize the training of a weight matrix (e.g., 105 or 107) for an accelerator (e.g., 111 or 121) of a type (e.g., 113 or 123) implemented using the synaptic memory cell array 145 as a computational element for multiplication, the loss function (e.g., 117 or 127) may be configured to penalize bits of a first type more than bits of a second type in the weights of the weight matrix (e.g., 105 or 127). For example, the bits of the first type have a value of 1; and the bits of the second type have a value of 0.

[0158] For example, adjusting the weight matrix at block 503 may include retraining the input weight matrix 106 according to a pruning selection (eg, 119 or 129 ) to suppress weight patterns in the input weight matrix 106 .

[0159] For example, retraining 104 may include modifying a first portion of input weight matrix 106 and adjusting a second portion of input weight matrix 106 to reduce a difference between an output generated using input weight matrix 106 and an output generated using a retrained weight matrix, such as 105 or 107. Retraining 104 may further include determining an accuracy performance level of the retrained weight matrix, such as 105 or 107, determining an energy performance level of the retrained weight matrix, such as 105 or 107, evaluating a combined performance level (e.g., by a weighted average) based on the accuracy performance level and the energy performance level, and searching for weight selection and modification solutions to improve or optimize the combined performance level of the retrained weight matrix, such as 105 or 107.

[0160] For example, adjusting the weight matrix at block 503 may include training the weight matrix (e.g., 105 or 107) according to the training data set 101 via machine learning 103. The training of the weight matrix (e.g., 105 or 107) may include reducing a loss function (e.g., 117 or 127) associated with the energy consumption characteristic (e.g., 115 or 125).

[0161] For example, to customize the training of a weight matrix (e.g., 105) for an accelerator of type 113 (e.g., 111) implemented using a microring resonator 143 as a computational element for multiplication, pruning selection 119 may be configured to select small weights from weight matrix 106 and increase the selected small weights.

[0162] For example, to customize the training of a weight matrix (e.g., 107) for an accelerator of type 123 (e.g., 121) implemented using memristors 147 as computational elements for multiplication, pruning selection 129 may be configured to select large weights from weight matrix 106 and reduce the selected large weights.

[0163] For example, to customize the training of a weight matrix (e.g., 105 or 107) for an accelerator (e.g., 111 or 121) of a type (e.g., 113 or 123) implemented using the synaptic memory cell array 145 as a computational element for multiplication, the pruning selection (e.g., 119 or 129) may be configured to select a first type of bit to convert a second type of bit in the weights in the weight matrix (e.g., 105 or 107). For example, the first type of bit has a value of 1; and the second type of bit has a value of 0.

[0164] At block 505, on a computing device (eg, such as Fig.12 A weight matrix (eg, 105 or 107) adjusted according to energy consumption characteristics is configured in FIG.

[0165] pass Figure 1 training or Figure 2 During retraining 104, the weight matrix (eg, 105 or 107) may have a weight pattern that is energy efficient for operation of the accelerator (eg, 100).

[0166] For example, when the accelerator (eg, 100) is a photon accelerator 133 having a microring resonator 143 as a computational element for multiplication, in a computational device (eg, Fig.12 The weight matrix (eg, 105) configured in FIG. 1 has a weight pattern with a weight distribution that is more concentrated in a first magnitude range than in a second magnitude range that is lower than the first magnitude range.

[0167] For example, when the accelerator (eg, 100) is an electric accelerator having the memristor 147 as a computing element for multiplication, in a computing device (eg, Fig.12 The weight matrix (eg, 105) configured in FIG. 1 has a weight form with a weight distribution that is more concentrated in the first magnitude range than in the second magnitude range that is lower than the first magnitude range.

[0168] For example, when the accelerator (eg, 100) is an analog computing module 135 having a synaptic memory cell array 145 as a computing element for multiplication, in a computing device (eg, Fig.12 A weight matrix (e.g., 105) configured in FIG. 1 has a weight pattern of weight bit distribution that is more concentrated in bits having a first value (e.g., 0) than in bits having a second value (e.g., 1).

[0169] At block 507, a computing device (e.g., Figure 2) performs calculations of the artificial neural network using a weight matrix (e.g., 105 or 107) configured in a computing device.

[0170] At block 509, an accelerator (eg, 100) of the type (eg, 113 or 123) accelerates multiplication and accumulation operations in computation of the artificial neural network.

[0171] In one embodiment, a set of instructions for causing the machine to perform any one or more of the methods discussed herein may be executed within an instance machine of a computer system. In some embodiments, the computer system may correspond to a host system that includes, is coupled to, or utilizes a memory subsystem or that can be used to perform the operations described above. In alternative embodiments, the machine may be connected (e.g., networked) to other machines in a LAN, an intranet, an extranet, the Internet, or any combination thereof. The machine may operate as a server or a client machine in a client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or a client machine in a cloud computing infrastructure or environment.

[0172] The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network appliance, a server, a network router, a switch or a bridge, a network attached storage facility, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. Furthermore, while a single machine is described, the term "machine" shall also be construed to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[0173] An example computer system includes a processing device, a main memory (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), static random access memory (SRAM), etc.), and a data storage system, which communicate with each other via a bus (which may include multiple buses).

[0174] The processing device represents one or more general-purpose processing devices, such as a microprocessor, a central processing unit, or the like. More specifically, the processing device may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor that implements other instruction sets, or a processor that implements a combination of instruction sets. The processing device may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or the like. The processing device is configured to execute instructions for performing the operations and steps discussed herein. The computer system may further include a network interface device for communicating over a network.

[0175] The data storage system may include a machine-readable medium (also referred to as a computer-readable medium) on which is stored one or more sets of instructions or software embodying any one or more of the methodologies or functions described herein. The instructions may also reside completely or at least partially in a main memory or in a processing device during execution by a computer system, the main memory and the processing device also constituting a machine-readable storage medium. The machine-readable medium, data storage system, or main memory may correspond to a memory subsystem.

[0176] In one embodiment, the instructions include instructions to implement functionality corresponding to the operations described above. Although the machine-readable medium is shown as a single medium in the example embodiment, the term "machine-readable storage medium" should be understood to include a single medium or multiple media that store one or more sets of instructions. The term "machine-readable storage medium" should also be understood to include any medium capable of storing or encoding a set of instructions for execution by a machine and causing the machine to perform any one or more methodologies of the present disclosure. Therefore, the term "machine-readable storage medium" should be understood to include, but not limited to, solid-state memory, optical media, and magnetic media.

[0177] Some portions of the foregoing detailed description have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the art of data processing to most effectively convey the substance of their work to others skilled in the art. An algorithm is herein and generally considered to be a self-consistent sequence of operations leading to a desired result. The operations are those requiring physical manipulation of physical quantities. Typically, although not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, combined, compared, and otherwise manipulated. Mainly for common purposes, it sometimes proves convenient to refer to these signals as bits, values, elements, symbols, characters, items, numbers, or the like.

[0178] It should be remembered, however, that all of these and similar terms are associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. The present disclosure may be directed to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage systems.

[0179] The present disclosure also relates to an apparatus for performing the operations described herein. This apparatus may be specially constructed for the intended purpose, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. This computer program may be stored in a computer-readable storage medium, such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs and magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of medium suitable for storing electronic instructions, each coupled to a computer system bus.

[0180] The algorithms and displays presented herein are not inherently related to any particular computer or other device. Various general purpose systems may be used with the programs according to the teachings herein, or it may prove convenient to construct more specialized equipment to perform the methods. The structure of a variety of these systems will occur as described in the following description. In addition, the present disclosure is not described with reference to any particular programming language. It will be appreciated that a variety of programming languages ​​may be used to implement the teachings of the present disclosure as described herein.

[0181] The present disclosure may be provided as a computer program product or software that may include a machine-readable medium having instructions stored thereon, which instructions may be used to program a computer system (or other electronic device) to perform a process according to the present disclosure. A machine-readable medium includes any mechanism for storing information in a form readable by a machine (e.g., a computer). In some embodiments, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium, such as a read-only memory ("ROM"), a random access memory ("RAM"), a magnetic disk storage medium, an optical storage medium, a flash memory component, etc.

[0182] In this description, various functions and operations are described as being performed or caused by computer instructions to simplify the description. However, those skilled in the art will recognize that such expressions mean that the functions result from the execution of computer instructions by one or more controllers or processors (e.g., microprocessors). Alternatively or in combination, the functions and operations may be implemented using dedicated circuitry (e.g., using an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA)) with or without software instructions. Embodiments may be implemented using hardwired circuitry without or in combination with software instructions. Therefore, the technology is neither limited to any specific combination of hardware circuitry and software, nor to any specific source of instructions executed by a data processing system.

[0183] In the foregoing description, embodiments of the present disclosure have been described with reference to specific example embodiments thereof. It will be apparent that various modifications may be made to the present disclosure without departing from the broader spirit and scope of the embodiments of the present disclosure as set forth in the appended claims. Accordingly, the description and drawings should be regarded in an illustrative rather than a restrictive sense.

Claims

1. A method comprising: Identify the type of accelerator for multiplication and accumulation operations; adjusting a weight matrix of the artificial neural network based on the energy consumption characteristics of an accelerator of the type described; In a computing device having an accelerator of the type described above, configuring the weight matrix adjusted according to the energy consumption characteristics; and Using the accelerator of the type described above, multiplication and accumulation operations in calculations of the artificial neural network performed using the weight matrix configured in the computing device are accelerated.

2. The method of claim 1, wherein the adjusting of the weight matrix comprises training the weight matrix based on a training data set.

3. The method of claim 2, wherein the training of the weight matrix comprises reducing a loss function associated with the energy consumption characteristic.

4. The method of claim 3, wherein the type of accelerator is implemented using a microring resonator as a computational element for multiplication; and the loss function is configured to penalize small weights more than large weights.

5. The method of claim 3, wherein the type of accelerator is implemented using memristors as computational elements for multiplication; and the loss function is configured to penalize large weights more than small weights.

6. The method of claim 3, wherein the type of accelerator is implemented using synaptic memory cells as computational elements for multiplication; and the loss function is configured to penalize bits of a first type more in weights than bits of a second type.

7. The method of claim 6, wherein the first type of bits have a value of 1; and the second type of bits have a value of 0.

8. The method of claim 1, wherein the adjustment of the weight matrix comprises retraining an input weight matrix according to a pruning selection to suppress weight patterns in the input weight matrix.

9. The method of claim 8, wherein the retraining comprises modifying a first portion of the input weight matrix and adjusting a second portion of the input weight matrix to reduce a difference between an output generated using the input weight matrix and an output generated using a retrained weight matrix.

10. The method of claim 9, wherein the retraining further comprises determining an accuracy performance level of the retrained weight matrix, determining an energy performance level of the retrained weight matrix, evaluating a combined performance level based on the accuracy performance level and the energy performance level, and searching for weight selection and modification solutions to improve or optimize the combined performance level.

Images