On-chip training of machine learning models

Abstract

Embodiments of the invention relate to on-chip training of machine learning models. In one example, a method (1000) includes providing first data to a machine learning model to produce second data (1002). The method further includes determining an error based on the second data and target second data (1004); and determining a loss gradient based on the error (1006). The method further includes updating the run sum by adding the gradient to the run sum of previous loss gradients (1008); and updating model parameters of the machine learning model based on the updated run sum (1010).

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 591,141, entitled “On-chip Machine Learning,” filed on October 18, 2023, which is incorporated herein by reference in its entirety. Technical Field

[0003] Embodiments of the present application relate to on-chip training of machine learning models. Background Art

[0004] A machine learning model can be trained to find patterns or make decisions from an input data set. An example of a machine learning model is an artificial neural network, which can have an architecture based on biological neural networks. Training a machine learning model can involve a large amount of memory and computing resources, which makes it challenging to train machine learning models on devices with limited memory and computing resources. Summary of the invention

[0005] In one example, a method includes providing first data to a machine learning model to generate second data. The method further includes: determining an error based on the second data and target second data; and determining a loss gradient based on the error. The method further includes: updating a running sum by adding the gradient to a running sum of previous loss gradients; and updating a model parameter of the machine learning model based on the updated running sum.

[0006] In one example, an integrated circuit includes a sensor interface, a memory, and a processor. The memory is configured to store data and instructions. The processor is configured to receive first data via the sensor interface. The processor is also configured to receive at least a subset of data representing a machine learning model, model parameters of the machine learning model, and a running sum of previous loss gradients from the memory. The processor is also configured to generate second data by providing the first data to the machine learning model. The processor is also configured to determine an error based on the second data and the target second data. The processor is configured to determine a loss gradient based on the error; update the running sum based on adding the loss gradient to the running sum; update the model parameters based on the updated running sum; and store the updated model parameters and the updated running sum in the memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 Schematic diagram illustrating a system that can perform inference operations according to some examples.

[0008] Figure 2 To support Figure 1Flowchart of an example operation sequence of an inference operation.

[0009] Figure 3 To be used for Figure 1 Illustration of an instance of a machine learning model with inference operations.

[0010] Figure 4 To support on-chip training operations Figure 1 A schematic diagram of an example of the internal components of a processor.

[0011] Figure 5 is a flow chart illustrating an example on-chip training operation.

[0012] Figure 6 Included are graphs illustrating examples of errors with respect to training rounds of an on-chip training operation.

[0013] Figure 7 To support on-chip training operations Figure 1 A schematic diagram of an example of the internal components of a processor.

[0014] Figure 8 To support on-chip training operations Figure 1 A schematic diagram of an example of the internal components of a processor.

[0015] Fig. 9 Contains graphs illustrating examples of errors for training rounds of on-chip training operations based on a running sum of previous loss gradients.

[0016] Fig.10 A flow chart illustrating an example of an on-chip training operation for a neural network is included.

[0017] Fig.11 A schematic diagram of a processor platform that includes examples of on-chip training operations and inference operations as described herein.

[0018] The same reference numbers are used throughout the drawings to indicate the same (or similar) features. DETAILED DESCRIPTION

[0019] Figure 1Schematic diagram for illustrating system 100. System 100 may include: a plurality of electronic devices 102, including electronic devices 102a, 102b, and 102c; and a cloud network 103. Each electronic device 102 may include a sensor 104 and a processor 106. For example, electronic device 102a includes sensor 104a and data processor 106a, electronic device 102b includes sensor 104b and data processor 106b, and electronic device 102c includes sensor 104c and data processor 106c. In some examples, sensor 104 and processor 106 may be part of an integrated circuit (IC). In some examples, processor 106 and sensor 104 may be located on different ICs, and processor 106 may have a sensor interface to couple to sensor 104.

[0020] The sensors 104 may be of various types, such as audio / acoustic sensors, motion sensors, image sensors, voltage and current sensors, etc. In some instances, each electronic device 102 may include multiple sensors of different types (e.g., acoustic sensors, motion sensors, voltage and current sensors) or multiple instances of the same type of sensor (e.g., multiple microphones). Each sensor system may receive a stimulus 108 (e.g., an acoustic signal, a light signal, an electrical signal, motion, etc.) and generate an output 110 based on the received stimulus. The output may indicate, for example, whether an event of interest is detected. For example, the electronic device 102a may generate an output 110a based on the stimulus 108a, the electronic device 102b may generate an output 110b based on the stimulus 108b, and the electronic device 102c may generate an output 110c based on the stimulus 108c.

[0021] In some examples, each electronic device 102 may be part of an Internet of Things (IoT) end node and may be part of an edge device of the network 103. Each electronic device 102 may also be attached or collocated with other devices. For example, each electronic device 102 may be attached to a motor to measure the voltage / current of the motor, and different electronic devices 102 may be attached to different motors. Each electronic device 102 may transmit its respective output 110 to the cloud network 103, which may perform additional operations based on the output (e.g., transmit an alert regarding a fault event, remotely disable a faulty device, etc.).

[0022] The processor 106 of a particular electronic device 102 may perform processing operations on signals (e.g., voltage / current measurements, audio data, image data, motion data, etc.) collected by the sensor 104 on the particular electronic device to generate an output 110. For example, in an instance where the sensor 104 includes a voltage / current sensor, the processor 106 may perform processing operations on a sequence of voltage / current signals, such as fault detection. In addition, in an instance where the sensor 104 includes an audio / acoustic sensor, the data processor 106 may perform processing operations such as keyword spotting, silence detection, and detection of specific acoustic features (e.g., broken glass, guns). In addition, in an instance where the sensor 104 includes a motion sensor, the data processor 106 may perform processing operations such as vibration detection, activity recognition, and anomaly detection (e.g., whether a window / door is knocked or opened when no one is home or at night). In addition, in an instance where the sensor 104 includes an image sensor, the data processor 106 may perform processing operations such as facial recognition, gesture recognition, and visual wake-up word detection (e.g., determining whether there is a person in the environment).

[0023] Processor 106 may process sensor signals generated by sensor 104 to generate output 110 . Figure 2 An example of an operational sequence 200 of the processor 106 is illustrated. Figure 2 , the processor 106 may receive a sensor signal 201 from the sensor 104, and perform a sampling operation 202 to generate samples, and perform an analog-to-digital conversion (ADC) operation 204 to convert the sampled signal into a digital value. The processor 106 may perform a preprocessing operation 206 to preprocess the digital value. In some instances, preprocessing may be performed to, for example, extract or emphasize features of the sensor signal that may facilitate subsequent processing operations. In some instances, the preprocessing operation may include a domain transformation operation, such as a transformation operation from a time domain to a frequency domain, to extract frequency components of the sensor signal. The frequency components of the sensor signal may provide more unique features to facilitate detection / non-detection of an event. In some instances, the preprocessing operation 206 may include a Fourier transform operation, such as an N-point fast Fourier transform (FFT) operation, performed on a batch of N samples of the sensor signal to generate frequency components of the sensor signal. For example, the preprocessing operation 206 may include a 2048-point FFT to process a batch of 2048 samples at a time.

[0024] After the preprocessing operation 206, the processor 106 may perform a processing operation 208 on the preprocessed sensor signal (e.g., FFT output) to provide an output 110. The processing operation may include a detection operation based on the sensed signal (e.g., fault detection, speech recognition, vibration detection, etc.). In some examples, the processor 106 may implement a machine learning model 210, such as an artificial neural network, which is trained to perform inference operations and / or classification operations on the preprocessed sensor signal to support the processing operation. As described below, model parameters of the machine learning model 210 may be updated in the training operation to increase the likelihood that the processing operation 208 provides a correct output, such as a correct detection of an event (e.g., a fault event, a security breach event, etc.) rather than an incorrect detection.

[0025] An artificial neural network (hereinafter referred to as a "neural network") may include multiple processing nodes. Examples of neural networks include autoencoders, deep neural networks (DNNs), convolutional neural networks (CNNs), and the like. Figure 3 An example of an autoencoder 300 that may be part of a machine learning model 210 is illustrated. Figure 3 , the autoencoder 300 may include an encoder section 302, a decoder section 304, and a bottleneck section 305 located between the encoder section 302 and the decoder section 304. Each of the encoder section 302, the decoder section 304, and the bottleneck section 305 includes one or more neural network layers. For example, the encoder section 302 includes neural network layers 306 and 308, the bottleneck section 305 includes a neural network layer 310, and the decoder section 304 includes neural network layers 312 and 314. As described below, the encoder section 302 and the decoder section 304 are mirror images of each other, wherein the decoder section 304 is configured to provide an output that matches the input to the encoder section 302.

[0026] Each neural network layer includes one or more processing nodes each configured to operate similarly to a neuron. For example, neural network layer 306 includes processing nodes 306_0, 306_2, ..., 306_n, neural network layer 308 includes processing nodes 308_1, 308_2, ..., 308_m, neural network layer 310 includes processing nodes 310_1 and 310_2, neural network layer 312 includes processing nodes 312_1, 312_2, ..., 312_m, and neural network layer 316 includes processing nodes 316_1, 316_2, ..., 316_n.

[0027] exist Figure 3In the example of , neural network layer 306 can be an input layer, neural network layer 314 can be an output layer, and neural network layers 308 and 312 can be intermediate layers. Neural network layer 306 can receive and process a batch of input data elements a0 (e.g., a0_0, a0_1, ..., a0_n) at a time, wherein each processing node of neural network layer 306 can receive the batch of input data elements. For example, if pre-processing operation 206 performs a 2048-point FFT and generates frequency components in 120 frequency bins at a time (in Figure 3 In some examples, the processing operation 208 may include a batch normalization operation 320 of the data elements in_1, in_2, ..., in_n, which includes re-centering and rescaling the data elements to generate input data elements a0_1, a0_2, ..., a0_n. The batch normalization operation 320 may improve the speed and stability of the training operation of the autoencoder 300.

[0028] Each processing node of the neural network layer 306 can receive a corresponding input data element in the batch (e.g., node 306_1 receives a0_1, node 306_2 receives a0_2, and node 306_n receives a0_n), and generate a set of intermediate output data elements by scaling the input data elements with a set of weight elements w1, one intermediate output data element for each node in nodes 308_1, 308_2, 308_n, etc.

[0029] The neural network layer 308 includes m number of processing nodes, where m is less than n. Each of the m number of processing nodes of the neural network layer 308 can receive the scaled input data elements from each of the n number of processing nodes of the neural network layer 306, and generate an intermediate output data element a1 (e.g., a1_1, a1_2, a1_m) by summing the scaled input data elements and adding a bias term b1 to the sum. For example, the processing node 308_1 can generate the intermediate output data element a1_1 as follows:

[0030]

[0031] In Equation 1, w1_i represents one of the weight element sets w1 used by the processing nodes of the neural network layer 306 to scale the corresponding input data element a0. For example, processing node 306_1 scales the input data element a0_1 with weight element w1_1, processing node 306_2 scales the input data element a0_2 with weight element w1_2, and processing node 306_n scales the input data element a0_n with weight element w1_n. In addition, b1_1 represents a bias term added to the sum of the scaled input data elements. Different processing nodes of the neural network layer 308 (e.g., processing nodes 308_2, 308_m, etc.) can receive input data elements scaled with different sets of weights w1 and add different bias terms b1 to the sum.

[0032] The neural network layer 310 of the bottleneck section 305 includes k number of processing nodes, where k is less than m. Each processing node of the neural network layer 310 can receive intermediate data elements from each of the m number of processing nodes of the neural network layer 308 (e.g., receiving a1_1 from processing node 308_1, receiving a1_2 from processing node 308_2, receiving a1_m from processing node 308_m, etc.), scale the intermediate data elements with a weight element set w2, sum the scaled intermediate data elements, and add a bias term b2 to the sum. Each of the neural network layers 310 can also process the sum with an activation function (e.g., a rectified linear unit function (ReLU)) to generate an intermediate data element a2. The activation function can introduce nonlinearity and simulate the operation of a neuron. For example, processing node 310_1 can generate an intermediate output data element a2_1 as follows:

[0033]

[0034] In Equation 2, w2_i represents one of the weight element sets w2 used by the processing nodes of the neural network layer 310 to scale the intermediate input data element a1_i output by one of the processing nodes of the neural network layer 308. For example, the processing node 310_1 scales the intermediate data element a1_1 from the processing node 308_1 with the weight element w2_1, scales the input data element a1_2 from the processing node 308_2 with the weight element w2_2, and scales the intermediate data element a1_m from the processing node 308_m with the weight element w2_m. In addition, b2_1 represents the bias term added to the sum of the scaled intermediate data elements. Different processing nodes of the neural network layer 310 (e.g., processing node 310_m, etc.) can scale the intermediate data element a1 with different sets of weights w2 to produce the sum, and add different bias terms b2 to the sum.

[0035] Neural network layer 312 includes m number of processing nodes and mirrors neural network layer 308. Each of the m number of processing nodes of neural network layer 312 can receive intermediate data element a2 from each processing node of neural network layer 310 (e.g., a2_1 from processing node 310_1, a2_k from processing node 310_k), scale the intermediate element with weight set w3, sum the scaled intermediate data elements, and add bias term b3 to the sum to generate intermediate data element a3 (e.g., a3_1, a3_2, a3_m). For example, processing node 312_1 can generate intermediate output data element a3_1 as follows:

[0036]

[0037] In Equation 3, w3_i represents one of the weight element sets w3 used by the processing nodes of the neural network layer 312 to scale the intermediate input data element a2_i output by one of the processing nodes of the neural network layer 310. For example, the processing node 312_1 scales the intermediate data element a2_1 from the processing node 310_1 with the weight element w3_1, and scales the intermediate data element a2_k from the processing node 310_k with the weight element w3_k. In addition, b3_1 represents the bias term added to the sum of the scaled intermediate data elements. Different processing nodes of the neural network layer 312 (e.g., processing nodes 312_2, 312_m, etc.) can scale the intermediate data element a2 with different sets of weight elements w3 to produce the sum, and add different bias terms b3 to the sum.

[0038] In addition, neural network layer 314 may be an output layer having n number of processing nodes, and the mirror image may be neural network layer 306 of the input layer. Each of the n number of processing nodes of neural network layer 314 may receive an intermediate data element a3 from each processing node of neural network layer 312 (e.g., a3_1 from processing node 312_1, a3_2 from processing node 312_2, a3_m from processing node 312_m), scale the intermediate element with a weight set w4, sum the scaled intermediate data elements, and add a bias term b4 to the sum to generate an intermediate data element a4. For example, processing node 314_1 may generate an intermediate output data element a4_1 as follows:

[0039]

[0040] In Equation 4, w4_i represents one of the weight element sets w4 used by the processing nodes of the neural network layer 314 to scale the intermediate input data element a3_i output by one of the processing nodes of the neural network layer 312. For example, the processing node 314_1 scales the intermediate data element a3_1 from the processing node 312_1 with the weight element w4_1, scales the intermediate data element a3_2 from the processing node 312_2 with the weight element w4_2, and scales the intermediate data element a3_m from the processing node 312_m with the weight element w4_m. In addition, b4_1 represents the bias term added to the sum of the scaled intermediate data elements. Different processing nodes of the neural network layer 314 (e.g., processing nodes 314_2, 314_n, etc.) can scale the intermediate data element a3 with different sets of weight elements w4 to produce the sum, and add different bias terms b4 to the sum.

[0041] The processing operation 208 may also include an inverse batch normalization operation 322 that is the inverse of the batch normalization operation 320 to convert the intermediate data element a4 into an output data element output (e.g., out_0, out_1, ..., out_n) as follows:

[0042] Output = inverse_batch_norm(a4) (Equation 5)

[0043] The encoder section 302 can be trained by adjusting the weight element sets w1 and w2 and the bias element sets b1 and b2 to extract the most relevant features from the input data in representing a specific inference result (e.g., normal operation of the device, the occurrence of a normal event, etc.), while removing irrelevant features. The extracted features, as represented by the intermediate outputs a1 and a2, have a lower dimension (and a small number of elements) than the batch of input data elements. Thus, a batch of input data elements a0 has n data elements, which are reduced by the neural network layer 308 to a set of m intermediate data elements a1, and the set of m intermediate data elements a1 is further reduced to a set of k intermediate data elements a2.

[0044] In addition, in the case where the encoder section 302 can extract more relevant features of the input data elements, the decoder section 304 can be trained by adjusting the weight element sets w3 and w4 and the bias element sets to reconstruct n output data elements out_0, out_1, ..., out_n that match the input data elements in_0, in_1, ..., in_n from the intermediate data element a2. Therefore, the neural network layer 312 reconstructs m intermediate data elements a3 from k intermediate data elements a2, the neural network layer 314 reconstructs n intermediate data elements a4 from m intermediate data elements a3, and the inverse batch normalization operation 322 generates n output data elements out_0, ..., out_n from the n intermediate data elements a4.

[0045] The weight element sets w1, w2, w3 and w4 and the bias element sets b1, b2, b3 and b4 of the autoencoder 300 can be trained together using an input data set containing features representing a specific inference result (e.g., normal operation of the device, the occurrence of a normal event, etc.). If a subsequent input data set also contains features representing a specific inference result, the difference between the input data elements and the output data elements of the trained autoencoder 300 can still be minimized. On the other hand, if a subsequent input data set contains features representing a different inference result (e.g., abnormal / faulty operation of the device, the occurrence of an abnormal event, etc.), the difference between the input data element and the output data element may exceed a threshold, which may indicate that the autoencoder 300 predicts a different inference result.

[0046] Therefore, the processing operation 208 can implement an error loss function 324, such as a mean square error (MSE) loss function, to calculate the difference (or error) between the input data element and the output data element. The processing operation 208 can also perform a comparison operation 326 to compare the error with a threshold value based on, for example, the mean (μ) and standard deviation (σ) of the training data. If the error exceeds the threshold value, the processing operation 208 can output a specific inference result indicating, for example, normal operation of the device, the occurrence of a normal event, etc. If the error does not exceed the threshold value, then the processing operation 208 can output a specific inference result indicating, for example, normal operation of the device, the occurrence of a normal event, etc.

[0047] As described above, the weight element sets w1, w2, w3, and w4 and the bias element sets b1, b2, b3, and b4 of the autoencoder 300 can be trained using an input data set containing features representing a specific inference result, wherein the goal of the training is to minimize the difference between the input data elements and the output data elements of the autoencoder 300. In one example, the autoencoder 300 can be trained using a backpropagation operation, wherein the error in the output of the output neural network layer (e.g., neural network layer 314) calculated using a loss function 324 (e.g., a mean square error (MSE) loss function) is determined, and the gradient of the loss / error relative to the weights and bias elements of the particular neural network layer is calculated. The error is also backpropagated to the previous neural network layer (e.g., neural network layer 312), and the propagated loss / error is relative to the gradient of the weights and bias elements of the previous neural network layer. For other previous neural network layers, the propagation of the error / loss and the calculation of the gradient are repeated. The weights and bias elements of the neural network layer are then updated based on the loss / error gradient of each neural network layer.

[0048] For example, in a training operation, training data including a set of input data elements (e.g., in_0, in_1, in_n) is input to the autoencoder 300 with a current set of weight elements w1, w2, w3, w4 and a set of bias elements b1, b2, b3, and b4. In some examples, the error or loss function L at the output layer (e.g., neural network layer 314) in terms of mean square error (MSE) can be calculated as follows:

[0049]

[0050] In Equation 5, n is the number of input data elements in the batch (and the number of processing nodes in the input layer), a4 is the output element of the neural network layer 314 and the input of the inverse batch normalization operation 322, and a0 is the output element of the batch normalization operation 320. The loss can define how well the autoencoder encodes a0, and then uses the current set of weight elements and bias elements to restore the most important features of a0 (same as a4), and the training operation is to update the weight elements and bias elements to minimize the loss so that the autoencoder can restore as many important features of a0 as possible.

[0051] The weights and biases of each neural network layer are updated as follows:

[0052]

[0053] In Equation 7, w represents the current weight element of the neural network layer, and w_n represents the updated weight element of the neural network layer. In addition, vtw represents the current weight update with momentum, m represents the momentum constant, η represents the learning rate, and represents the partial derivative of the loss with respect to the weight element of the neural network layer. In addition, in Equation 8, b represents the current bias element of the neural network layer, and b_n represents the updated bias element of the neural network layer. In addition, vtb represents the current bias update with momentum, and represents the partial derivative of the loss with respect to the bias element of the neural network layer. For neural network layer 314, the gradient of the loss with respect to the bias is as follows:

[0054]

[0055] And the loss gradient with respect to the weight (w4) is as follows:

[0056]

[0057] In equations 9 and 10, e represents the error between the intermediate data element a4 and its target / reference at the neural network layer 314. In addition, the loss gradient with respect to the weight is a function of the input to the neural network layer 314 (the intermediate data element a3 from the neural network layer 312).

[0058] And the weight element w4 and the bias element b4 are updated as follows:

[0059] w4_n=w4-[vtw*m+η*a3*e] (Equation 11)

[0060] b4_n=b4-[vtb*m+η*e] (Equation 12)

[0061] In Equation 11, a3*e represents the loss gradient of the weight element calculated from the product of the input data of the neural network layer (e.g., a3 for neural network layer 314) and the instantaneous error at the neural network layer (e). In Equation 12, e(error) also represents the loss gradient (partial derivative) of the bias element.

[0062] The training operation for the previous neural network layer, such as neural network layers 312, 310, 308, and 306, may be repeated by back-propagating the error based on equations 5 to 12 above. For example, for neural network layer 312, the target output a4t at neural network layer 314 may be used to calculate the target input (a3t) of neural network layer 312 based on the current set of weights and bias elements (w3 and b3) of neural network layer 312, which is also the target output of neural network layer 312. The error e at neural network layer 312 may then be calculated based on the difference between the target output (a3t) of neural network layer 312 and the actual output (a3) ​​of neural network layer 312. New weights and bias elements (w3_n and b3_n) of neural network layer 312 may be calculated based on equations 11 and 12, where a3 is replaced by a2 in equation 11.

[0063] In some examples, the machine learning model 210 may include a deep neural network (DNN) having multiple neural network layers, the multiple neural network layers including an input layer, an output layer, and one or more hidden layers located between the input layer and the output layer. Each neural network layer includes a processing node, and each processing node can scale the input data with a weight set based on equations 1 to 4 above, sum the scaled input data, and add a bias term to the sum to simulate the operation of a neuron. However, compared to the autoencoder 300, the hidden layer does not form a bottleneck section with a reduced size / dimension compared to the input layer and the output layer. In addition, the DNN can be configured as a classification network to classify the input into one of a plurality of categories. The output of the DNN can represent, for example, the probability of the input belonging to each of a plurality of classification categories, and the processing operation 208 can output the category with the highest probability. For example, the DNN can output a first probability that the input is classified as representing the normal operation of the device and a second probability that the input is classified as representing the abnormal / faulty operation of the device. If the first probability is higher than the second probability, the processing operation 208 can output an indication that the normal operation of the device is detected. However, if the second probability is higher than the first probability, then processing operation 208 may output an indication that abnormal / faulty operation of the device has been detected.

[0064] The DNN can also be trained using the aforementioned back-propagation operation, but using a loss function defined based on cross entropy rather than mean squared error. The training data for the DNN can also include input data representing different classification categories, as well as target category labels for the input data. The goal of the training is to minimize the cross entropy loss function, which represents the mismatch between the category output by the DNN and the target category (e.g., represented by a4t). Updating the weights and bias elements of the neural network layer of the DNN based on the cross entropy loss gradient can also be based on the above equations 11 and 12.

[0065] Reference again Figure 1In some examples, the data processor 106 of each electronic device 102 may receive data 120 (e.g., 120a, 120b, and 120c) from the cloud network 103, the data including, for example, compiled instructions representing each layer of the neural network model and model parameters of the neural network model, such as a set of weight elements and a set of bias elements. In some examples, the cloud network 103 may perform the aforementioned back-propagation training operation of the neural network model to generate or update the model parameters. As part of the training operation, the cloud network 103 may receive outputs 110 (e.g., outputs 110a, 110b, and 110c) generated by the electronic device 102 and sensor signals generated by the sensor 104 of the electronic device 102 in response to stimuli sensed by the sensor 104 (e.g., stimuli 108a, 108b, and 108c). The cloud network 103 may also receive target / reference outputs for these stimuli and adjust the model parameters to reduce / minimize errors in the outputs of the neural network model (relative to the reference / target outputs) when processing the sensor signals representing the stimuli. After the training operation is completed, the cloud network 103 may transmit the new model parameters back to the electronic device 102. The electronic device 102 may then perform an inference operation using the neural network model with the new model parameters.

[0066] As described above, the training operations performed by the cloud network 103 are based on stimuli received by the electronic devices 102, their outputs in response to the stimuli, and the target / reference outputs of the electronic devices 102 to those stimuli. Such an arrangement can take into account the local operating conditions of each individual electronic device 102, which can improve the robustness of the training operations. However, when some of the electronic devices have very different operating conditions, the robustness may be limited. Assuming that all electronic devices 102 share the same machine learning model and the same model parameters, those model parameters may be optimal for some of the electronic devices (e.g., achieving the lowest loss), but not optimal for all electronic devices.

[0067] Performing training operations at the cloud network 103 may also impose limitations on the scalability and frequency of training operations. Specifically, as more electronic devices 102 are added to the system, existing electronic devices 102 are reconfigured (e.g., installed on new motors, sensors are replaced), etc., the machine learning model can be retrained to consider the set of new operating conditions of the additional electronic devices 102, but as more input data and output data of the additional electronic devices are input to the training operation, the complexity of the training operation increases, which may increase the duration of the training operation, as well as the computing and memory resources used for the training operation. Due to the increased duration and resource usage of the training operation, the machine learning model also cannot be trained frequently, which may make it difficult to adapt the machine learning model to changes in the operating conditions of the electronic device 102.

[0068] One way to address this type of problem is by performing on-chip training, where the training operation of the machine learning model occurs locally at the electronic device 102. Figure 1 , each of the electronic devices 102a, 102b, and 102c may receive compiled instructions representing each layer of the neural network model, as well as initial model parameters of the neural network model. And then each of the electronic devices 102a, 102b, and 102c (e.g., processor 106) may locally perform a training operation using the stimulus received by each electronic device to update the model parameters. For example, the processor 106a of the electronic device 102a may perform a local training operation using the stimulus 108a sensed by the sensor 104a, the processor 106b of the electronic device 102b may perform a local training operation using the stimulus 108b sensed by the sensor 104b, and the electronic device 102c may perform a local training operation using the stimulus 108c sensed by the sensor 104c. Therefore, when the electronic devices 102 share the same machine learning model, the machine learning model parameters may differ between the devices.

[0069] On-chip training operations can provide various advantages. Specifically, because the training operation occurs locally at each electronic device 102, the training operation can become distributed and naturally scalable with the number of electronic devices 102, and the complexity of the training operation does not increase with the number of electronic devices 102. This also allows training to be performed more frequently (for example, when the electronic device 102 is reconfigured in response to a loss exceeding a threshold, etc.) to adapt the machine learning model to changes in the operating conditions of the electronic device 102. In addition, because each electronic device 102 can perform local training operations that take into account local operating conditions, the model parameters generated from the local training operation may be optimal for each electronic device. In addition, because each electronic device 102 does not need to transmit the input and output data of the machine learning model to the cloud network 103 to support the training operation, the data transmission between the cloud network 103 and the electronic device 102 can be reduced, which not only reduces power, but also reduces the privacy / security risks associated with data transmission.

[0070] In some examples, one on-chip training round may involve a single batch of input data, where model parameters (e.g., weights and bias elements) are updated based on the single batch of input data. Such an arrangement may reduce memory usage for training operations, which may facilitate implementation of on-chip training on devices with limited memory resources.

[0071] Figure 4 An example of internal components of processor 106 used to support on-chip training operations is described. Figure 4, the processor 106 may include an inference module 402 and a back propagation module 404. Each module may include circuitry and / or software instructions. For example, as described below, the processor 106 may include an execution unit to execute the compiled instructions of each module. The processor 106 may also include arithmetic circuitry (e.g., a matrix multiplication accelerator (MMA)) to perform the calculations described in some of the aforementioned equations.

[0072] refer to Figure 4 , the inference module 402 may extract a set of input data 406 and current model parameters 408 (e.g., weights and bias elements). The input data and model parameters may be extracted from a memory 409, which may be internal or external to the processor 106. Figure 4 In the example, input data 406 may represent a batch of input data (e.g., frequency components in 120 frequency points) from preprocessing operation 206, or intermediate data generated by the aforementioned neural network layer based on the batch of input data. Inference module 402 may provide data and parameters to neural network layer 410 to generate output data 412 of neural network layer 410. For example, in neural network layer 410 is Figure 3 In the case of the neural network layer 314, the inference module 402 can extract the weight element w4 and the bias element b4 as part of the model parameter 408, and calculate the intermediate data a4 as the output data 412 based on the above equation 4. In some examples, the calculation can be performed using MMA.

[0073] The back propagation module 404 may include a loss gradient calculation module 414 and a model parameter update module 416. Referring back to the above equation 6, the loss gradient calculation module 414 may calculate the target output data of the neural network layer 410 based on the output data 412, calculate the error (e) between the target output data of the neural network layer 410 and the output data 412, and calculate the loss gradient of the weight and bias elements (e.g., w4 and b4) based on the error e according to equations 9 and 10. The model parameter update module 416 may calculate a new set of model parameters (e.g., w4_n and b4_n) based on the loss gradient according to equations 11 and 12, and update the model parameters 408 in the memory 409 with the new set of model parameters. The processor 106 may also propagate the error back to the previous neural network layers (e.g., neural network layers 312, 310, 308, and 306), and calculate the error at each of the previous neural network layers, and adjust the weight and bias elements of each neural network layer based on the error.

[0074] Figure 5An example of on-chip training operations of an autoencoder (eg, autoencoder 300 ) based on a single batch of input data that may be performed by the inference module 402 and the back-propagation module 404 of the processor 106 is described. Figure 5 , the inference module 402 can use the first batch of input data 406 (e.g., the first set of frequency components in 120 frequency points) to perform an inference operation 502 to generate a first output data set 506 (e.g., Figure 3 , out_0, out_1, ..., out_n in ). The back propagation module 404 can then use the first output data set 406 to perform a first round of training operations 508. For example, referring back to the above equation 6, the back propagation module 404 can calculate the target output data of the neural network layer 314 based on the output data 406, calculate the error (e) between the target output data of the neural network layer 314 and the output data 406, calculate the loss gradient of the weight and bias elements (w4 and b4) based on the error e according to equations 9 and 10, and then adjust the weight element w4 and the bias element b4 of the neural network layer 314 based on the loss gradient according to equations 11 and 12. The back propagation module 404 can also propagate the error back to the previous neural network layers (e.g., neural network layers 312, 310, 308, and 306), and calculate the error at each of the previous neural network layers, and adjust the weight and bias elements of each neural network layer based on the error.

[0075] After the round of training operation 508 is completed and the model parameters are updated, the inference module 402 can use the second batch of input data 514 (e.g., a second set of frequency components in 120 frequency points) to perform another inference operation 512 to generate a second output data set 516 (e.g., Figure 3 The back propagation module 404 can then use the second output data set 516 to perform a second round of training operations 518 and update the weights and bias elements of the machine learning model. Therefore, the weights and bias elements of the machine learning model are updated according to the input data batch.

[0076] Figure 5 The arrangement can reduce the memory usage of on-chip training operations. Specifically, for each training round, memory resources are provided to store a single batch of input data and intermediate data and output data of the neural network layer (from inference operations) and target intermediate data and output data (for error calculation) generated from the single batch of input data of the neural network layer. After the training round is completed, the memory can be overwritten with a new single batch of input data and a new set of intermediate data, output data and their targets generated from the new single batch of input data. Such an arrangement can reduce the total memory usage of the data involved in the training round, which can facilitate the implementation of on-chip training on devices with limited memory resources.

[0077] although Figure 5 On-chip training operations (which update model parameters based on batches of input data) can reduce memory usage, but the robustness of on-chip training operations may decrease when the input signal exhibits rapid and large changes over time (e.g., transients). Errors (and losses) also contain transients. Such rapid and large transients in input signals (and errors) may not provide useful features for detecting / not detecting events, and may exist in some batches of input data and may not exist in other batches of input data. If the batch of input data provided to the previous round of training operations does not contain such transients, and a new batch of input data provided to a new round of training operations contains such transients, then the new round of training operations may calculate large (and incorrect) errors and large (and incorrect) loss gradients based on the new batch of input data. This may lead to gradient explosion.

[0078] Figure 6 600 is included to illustrate an example of gradient explosion caused by an on-chip training operation, where the loss gradient is calculated based on a single batch of input data. Figure 6 As shown in FIG. 1 , in each training round from round number 10 to round number 40, the mean square error (MSE) calculated in each training round is in a narrow range of 0.01 to 2. However, after round number 40, due to the presence of transients in the input signal, the MSE jumps to 10. 12 The significant errors (and loss gradients) are caused by the absence of those transients in the input signals of training rounds No. 10 to No. 40. If those transients do not provide useful features for detecting events (e.g., faulty operation of the device, abnormal events, etc.), the training operation may adjust weight and bias elements based on erroneous loss gradients, which reduces the robustness of the on-chip training operation.

[0079] In addition, stability issues may arise when the training operation combines a large learning rate (η in Equations 11 and 12) with a large loss gradient. This is because in one training round with a batch of input data with transients, the training operation can adjust the model parameters in one direction based on the large loss gradient. And then in subsequent training with another batch of input data without transients (or with transients in the opposite direction), the training operation can calculate another large loss gradient again, and adjust the model parameters again in a large amount but in the opposite direction. Because the large loss gradient persists (but in the opposite direction), the training operation can continue to adjust the model parameters in the opposite direction in subsequent rounds, which may cause the model parameters to be unstable.

[0080] The gradient explosion problem can be alleviated in various ways, but each has its own disadvantages. For example, in order to improve stability, the training operation can be configured to use a reduced learning rate. But such an arrangement may cause the training to converge slowly. As another example, instead of adjusting the model parameters according to a training operation using a single batch of input data, the model parameters can be adjusted by a training operation using multiple batches of input data. For example, multiple loss gradients can be calculated based on multiple batches of input data with the same current model parameter set. The loss gradient can be averaged to reduce the huge loss gradient caused by transients, and the current model parameter set can then be adjusted using the averaged gradient. However, such an arrangement may require a large amount of memory resources to store multiple batches of input data and intermediate and output data generated from multiple batches of input data for averaging, and may not be suitable for devices with limited memory resources.

[0081] Figure 7 Examples of components of the processor 106 that may address at least some of the issues described above are described. Figure 7 , the model parameter update module 416 may include a loss gradient accumulation module 704, which may maintain a running sum 702 of previous loss gradients in the memory 409. During training operations, the loss gradient calculation module 414 may calculate an instantaneous / latest loss gradient based on an instantaneous error e (e.g., a3*e of weight element w4 and e of bias element b4 for the neural network layer 314). The loss gradient accumulation module 704 may add the latest loss gradient to the running sum 702 of previous loss gradients calculated over multiple previous training rounds.

[0082] The running sum may be updated in each training round, where the loss gradient is calculated based on a batch of input data. For example, for the neural network layer 314, the loss gradient may be a3·e of the weight element w4 and e of the bias element b4, where e is the error between the actual output a4 of the neural network layer 314 and the target output a4t. In different training rounds, the running sum 702 of the previous loss gradients may include the running sum of a3·e of the weight w4 of the neural network layer 314 and the running sum of e of the bias b4, as well as the running sums of other loss gradients of other neural network layers. The running sum may smooth (e.g., low-pass filter) large transients in the input data and large transients in the loss gradient over time, which may prevent or at least reduce the aforementioned gradient explosion problem. On the other hand, the running sum may reflect long-term (e.g., static) errors, and the back-propagation module 704 may adjust the weight / bias elements to reduce the running sum to zero so that the long-term error may also be close to zero. These long-term errors can indicate persistent features / patterns in the input data that the machine learning model has not properly accounted for during inference operations, rather than transient features. Therefore, adjusting weight / bias elements to eliminate long-term errors can improve the robustness of the machine learning model during inference operations.

[0083] In some examples, the loss gradient calculation module 414 may also calculate the average error over a sliding window of training rounds including the latest training round Based on the average error (For example, for neural network layer 314, the weight element w4 and the deviation element b4 ) and determine the loss gradient, and the loss gradient accumulation module 704 can add the loss gradient to the running sum 702 of the previous loss gradients calculated over multiple previous sliding windows. Such an arrangement can further reduce the huge loss gradient caused by transients and alleviate the gradient explosion problem.

[0084] In some examples, the loss gradient accumulation module 704 can also clamp the running sum (e.g., by resetting it to zero, by dividing the running sum by the number to obtain an average, etc.). In some examples, the loss gradient accumulation module 704 can clamp the running sum in response to the running sum exceeding a threshold to prevent overflow.

[0085] The model parameter update module 416 can generate update parameters based on the updated running sum, and adjust the model parameters (e.g., weight elements and bias elements) based on the update parameters. Figure 3 The loss gradient accumulation module 704 may generate update parameters based on the running sum, and the model parameter update module 416 may update the weight element w4 and the bias element b4 as follows:

[0086] w4_n=w4-[∫a3·edt] (Equation 13)

[0087] b4_n = b4 - [∫e dt] (Equation 14)

[0088] In Equation 13, ∫a3·edt is the integral of a3·e, which represents the running sum of the loss gradient a3·e of the weight element w4. The update parameter of the weight element w4 is generated based on the running sum of the loss gradient a3·e. In addition, in Equation 14, ∫e dt represents the running sum of the loss gradient e of the bias element b4. The update parameter of the bias element b4 is generated based on the running sum of the loss gradient e.

[0089] Figure 8 Additional examples of components of processor 106 are described. Figure 8 , the model parameter update module 416 may include a proportional and integral (PI) controller 802. The proportional and integral (PI) controller 802 may generate a proportional adjustment parameter and an integral adjustment parameter based on the latest loss gradient calculated by the loss gradient calculation module 414. Specifically, the proportional adjustment parameter may be based on the latest loss gradient, which allows the back propagation module 704 to quickly respond to transients and reduce errors caused by transients by adjusting weight / bias elements, which may improve system stability and reduce the likelihood of system oscillations (e.g., weights and biases oscillating around certain values ​​between training rounds). In addition, the integral adjustment parameter is based on a running sum 702 of previous loss gradients, which may smooth large transients in the loss gradient over time and prevent or at least reduce the aforementioned gradient explosion problem.

[0090] For example, for Figure 3 The neural network layer 314 of , the proportional and integral controller 802 can generate a proportional update parameter based on the latest loss gradient and an integral update parameter based on the updated running sum 702 of the previous loss gradients, and the model parameter update module 416 can update the weight element w4 and the bias element b4 as follows:

[0091] w4_nw4-[Kpw*a3*e+K iw ∫a3*e dt] (Equation 15)

[0092] b4_n=b4-[K pb *e+K ib ∫e dt] (Equation 16)

[0093] In Equation 15, K pw a3 e represents the proportional adjustment parameter of the weight element w4, and K iw∫a3·edt represents the integral adjustment parameter of the weight element w4. In addition, in Equation 16, K pb ·e represents the proportional adjustment parameter of the deviation element b4, and K ib ∫e dt represents the integral adjustment parameter of the weight element b4. K pw , K iw , K pb and K ib may be part of the PI parameters 804 in memory 409 and may be part of the hyperparameters of the machine learning model. In some examples, the instantaneous error e in equations 15 and 16 may be the average error as explained above. Presentation.

[0094] Fig. 9 Contains information about MSE relative to the Figure 7 and 8 900 is a graph of an example of changes in the number of training rounds performed by the back-propagation module 404 of FIG. Fig. 9 As shown in , MSE starts with a relatively high value (e.g., 500) in the first training round, but as training proceeds, MSE converges to a low value (e.g., 20) with a flat slope, and can avoid Figure 6 The exploding gradient problem shown in .

[0095] Fig.10 Flow chart illustrating a method 1000 for training a neural network. The method 1000 may be performed by, for example, the inference module 402 and the back propagation module 404 of the examples described herein. Fig.10 In operation 1002, the inference module 402 may provide the first data to a machine learning model (e.g., a neural network layer 410) to generate second data. The machine learning model may include, for example, an autoencoder, a DNN, etc. In some instances, the first data may be a single batch of input data, or intermediate data generated by a previous neural network layer from the single batch of input data, where a single batch represents the number of input data elements that the machine learning model can process at one time. A single batch may be based on, for example, the number of processing nodes of an input layer of a machine learning model. In some instances, the input data comes from a Fourier transform preprocessing operation (e.g., an FFT operation), and a single batch may be based on the number of frequency points output by the FFT operation.

[0096] In operation 1004, the back propagation module 404 may determine an error based on the second data and the target second data. The determination of the error may be based on a loss function. For an autoencoder, the target second data may be calculated based on the second data, for example based on equation 6. For a DNN, the target second data may be provided externally as reference data (e.g., expected classification output). For an autoencoder, the loss function may be based on a mean square error (MSE) between the second data and the target second data. For a DNN (or classification network), the loss function may be based on a cross entropy.

[0097] In operation 1006, the back propagation module 404 may determine a loss gradient based on the error. The back propagation module 404 may determine a loss gradient with respect to the weight element and a loss gradient with respect to the bias element based on equations 9 and 10.

[0098] In operation 1008, the back propagation module 404 may update the running sum by adding the loss gradient determined in operation 1006 to the running sum of the previous loss gradients according to equations 13 and 14. The running sum may be an integral operation to smooth / low-pass transients in the loss gradients, and may be updated per training round and based on a batch of input data. In some examples, if the running sum exceeds a threshold, the back propagation module 404 may limit the running sum (e.g., reset to zero, divide the running sum by a number to obtain an average, etc.) to prevent overflow.

[0099] In operation 1010, the back propagation module 404 can update the model parameters of the machine learning model based on the updated running sum. In some examples, the back propagation module 404 can include a PI controller, such as the PI controller 802, to generate a proportional adjustment parameter and an integral adjustment parameter, and adjust the weight element and the bias element based on the proportional adjustment parameter and the integral adjustment parameter according to equations 15 and 16.

[0100] Fig.111 is a block diagram of an example processor platform 1100 including a processor circuit system configured to execute machine-readable instructions. The processor platform 1100 may be part of the processor 106 or may include the processor 106. The processor platform 1100 of the illustrated example may include a processor circuit system 1112. The processor circuit system 1112 of the illustrated example includes hardware. For example, the processor circuit system 1112 may be implemented by one or more integrated circuits, logic circuits, FPGAs, microprocessors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), and / or microcontrollers from any desired series or manufacturer. The processor circuit system 1112 may be implemented by one or more semiconductor-based (e.g., silicon-based) devices. The processor circuit system 1112 may execute compiled instructions, such as instructions representing neural network layers and their calculations. In some examples, the processor platform 1100 may also include a machine learning hardware accelerator 1122, which may include a matrix multiplication accelerator (MMA) and other circuit systems to facilitate neural network calculations. The machine learning hardware accelerator 1122 may be external to or may be part of the processor circuitry 1112. The processor circuitry 1112 and the machine learning hardware accelerator 1122 together may implement, for example, the inference module 402 and the back-propagation module 404.

[0101] The processor circuit system 1112 of the illustrated example may include local memory 1113 (e.g., cache, registers, etc.). The local memory 1113 may be an example of the memory 409 and may store, for example, the model parameters 408, the input data and output data of each neural network layer, and the running sum of the previous loss gradients 702. The processor circuit system 1112 of the illustrated example communicates with a computer-readable storage device, such as a main memory including volatile memory 1114 and non-volatile memory 1116, via a bus 1118. The volatile memory 1114 may be composed of, for example, synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), Dynamic Random Access Memory The non-volatile memory 816 may be implemented by a programmable read-only memory, a flash memory, and / or any other desired type of non-volatile memory device. Access to the main memory 1114, 1116 of the illustrated example may be controlled by a memory controller 1117.

[0102] The processor platform 1100 of the illustrated example also includes a network interface circuit system 1120 for connecting to a network 1126 (e.g., cloud network 103). The network interface circuit system 1120 can be implemented by hardware that conforms to any type of interface standard, such as an inter-integrated circuit (I2C) interface, a serial peripheral interface (SPI), an Ethernet interface, a universal serial bus (USB) interface, interface, a near field communication (NFC) interface, a peripheral component interconnect (PCI) interface, and / or a peripheral component interconnect express (PCIe) interface.

[0103] The processor platform 1100 of the illustrated example also includes a sensor interface 1130, through which the processor platform 1100 can receive sensor signals (e.g., voltage and current measurement signals, acoustic signals, vibration signals, etc.) from, for example, the sensor 104. The processor platform 1100 also includes an analog-to-digital converter (not shown) to convert analog signals into digital signals for processing by the processor circuit system 1112.

[0104] The machine-readable instructions 1132 may be stored in the volatile memory 1114 and / or the non-volatile memory 1116. When executed by the processor circuit system 812, the machine-readable instructions 1132 cause the processor platform 1100 to perform any or all of the functions described herein as being attributed to the processor 106, such as the method 1000. The instructions may represent a machine learning model and an application that invokes the machine learning model to perform inference operations on sensor signals received via the sensor interface 1130.

[0105] In this specification, the term "coupled" may encompass connections, communications, or signal paths that enable a functional relationship consistent with this specification. For example, if device A provides a signal to control device B to perform an action, then: (a) in a first instance, device A is coupled to device B; or (b) in a second instance, if the intermediate component C does not substantially change the functional relationship between device A and device B, then device A is coupled to device B through the intermediate component C, so that device B is controlled by device A via the control signal provided by device A. In addition, in this specification, a device "configured to" perform a task or function may be configured (e.g., programmed and / or hardwired) to perform the function when manufactured by a manufacturer, and / or may be configured (or reconfigurable) by a user after manufacturing to perform the function and / or other additional or alternative functions. Configuration may be performed by firmware and / or software programming of the device, by the construction and / or layout of the hardware components and interconnects of the device, or a combination thereof. In addition, in this specification, a circuit or device containing certain components may actually be used to couple to those components to form the described circuit system or device. For example, a structure described as including one or more semiconductor elements (e.g., transistors), one or more passive elements (e.g., resistors, capacitors, and / or inductors), and / or one or more sources (e.g., voltage sources and / or current sources) may actually include only semiconductor elements within a single physical device (e.g., a semiconductor die and / or integrated circuit (IC) package), and may be suitable for coupling to at least some of the passive elements and / or sources to form the described structure during or after manufacturing, for example, by an end user and / or a third party.

[0106] Although specific transistor structures are mentioned above, other transistors or device structures may be used instead. For example, a p-type MOSFET may be used in place of an n-type MOSFET with little or no additional changes. In addition, other types of transistors (e.g., bipolar transistors) may be used in place of the transistors shown. The capacitor may be implemented using a different device structure (e.g., metal structures formed on top of each other to form a parallel plate capacitor), or may be formed on a layer (metal or doped semiconductor) that is closer to or further from the surface of the semiconductor substrate.

[0107] As used above, the terms "terminal", "node", "interconnection" and "pin" are used interchangeably. Unless specifically stated to the contrary, these terms are generally used to refer to the interconnection between device elements, circuit elements, integrated circuits, devices or other electronic devices or semiconductor components or their ends.

[0108] Although some components may be described herein as belonging to a specific process technology, these components may be exchanged with components of other process technologies. The circuits described herein may be reconfigured to include replaced components to provide functionality at least partially similar to functionality available before component replacement. Unless otherwise stated, components shown as resistors generally represent any one or more elements coupled in series and / or in parallel to provide the impedance amount represented by the displayed resistor. For example, a resistor or capacitor shown and described as a single component herein may actually be a plurality of resistors or capacitors coupled in series or in parallel between two nodes identical to a single resistor or capacitor, respectively. In addition, the use of the phrase "ground terminal" in this specification includes chassis grounding, ground wire grounding, floating grounding, virtual grounding, digital grounding, public grounding, and / or any other form of ground connection applicable to or suitable for the teachings of this specification. Unless otherwise stated, "about", "roughly" or "substantially" in front of a value means + / -10% of the stated value.

[0109] Modifications are possible in the described examples, and other examples are possible, within the scope of the claims.

Claims

1. A method comprising: providing the first data to a machine learning model to generate second data; determining an error based on the second data and target second data; determining a loss gradient based on the error; updating the running sum by adding the loss gradient to a running sum of previous loss gradients; and Model parameters of the machine learning model are updated based on the updated running sum.

2. The method of claim 1 , wherein updating model parameters of the machine learning model based on the running sum comprises updating model parameters of the machine learning model based on a combination of the loss gradient and the updated running sum. 3 . The method of claim 2 , further comprising determining the combination using a proportional-integral controller.

4. The method of claim 3, wherein determining the combination comprises using the proportional-integral controller to: multiplying the loss gradient by a proportional gain parameter to generate a proportional adjustment parameter; multiplying the updated running total by an integral gain parameter to generate an integral adjustment parameter; and The combination is generated based on the proportional adjustment parameter and the integral adjustment parameter. The method of claim 1 , further comprising limiting the running sum.

6. The method of claim 1, wherein the machine learning model comprises a neural network model.

7. The method of claim 6, wherein the model parameters include weight elements, and the loss gradient is determined based on the first data and the error.

8. The method of claim 6, wherein the model parameters include a bias parameter and the loss gradient is determined based on the error.

9. The method of claim 6, wherein the neural network model comprises at least one of: a convolutional neural network, a deep neural network, or an autoencoder.

10. The method of claim 1, wherein the machine learning model processes a batch of input data at a time, and the first data comprises a single batch of the input data.

11. The method of claim 1 , further comprising updating the machine learning model based on batches of input data.

12. The method of claim 1, wherein the first data is provided by a sensor, the method is performed by a device comprising the sensor, and the previous loss gradient is generated by previous first data from the sensor.

13. The method of claim 1, further comprising performing a fault detection operation based on the second data.

14. An integrated circuit comprising: Sensor interface; a memory configured to store data and instructions; and A processor configured to execute the instructions to: receiving first data via the sensor interface; receiving from the memory at least a subset of the data representing a machine learning model, model parameters of the machine learning model, and a running sum of previous loss gradients; generating second data by providing the first data to the machine learning model; determining an error based on the second data and target second data; determining a loss gradient based on the error; updating the running sum based on adding the loss gradient to the running sum; updating the model parameters based on the updated running sum; and Updated model parameters and the updated running sum are stored in the memory.

15. The integrated circuit of claim 14, wherein the machine learning model is configured to process a batch of input data at a time, and the first data comprises a single batch of the input data.

16. The integrated circuit of claim 14, wherein the processor is configured to execute the instructions to update the model parameters of the machine learning model based on a combination of the loss gradient and the updated running sum.

17. The integrated circuit of claim 16, wherein the processor is configured to execute the instructions to implement a proportional-integral controller and to determine the combination using the proportional-integral controller.

18. The integrated circuit of claim 17, wherein the processor is configured to execute the instructions to use the proportional-integral controller to: multiplying the loss gradient by a proportional gain parameter to generate a proportional adjustment parameter; multiplying the updated running total by an integral gain parameter to generate an integral adjustment parameter; and The combination is generated based on the proportional adjustment parameter and the integral adjustment parameter.

19. The integrated circuit of claim 14, wherein the processor comprises a machine learning hardware accelerator.

20. The integrated circuit of claim 14, wherein the machine learning model comprises at least one of: a convolutional neural network, a deep neural network, or an autoencoder.

21. The integrated circuit of claim 14, wherein the processor is configured to execute the instructions to provide a fault detection indication based on the second data.

Images