Intermittent Deep Neural Network Inference Method for Energy Harvesting Internet of Things

Abstract

This application discloses an intermittent deep neural network inference method for energy harvesting IoT, specifically relating to the field of neural network inference methods. It includes: retrieving input data, model parameters, and progress tracking variables for each layer of a deep neural network model from non-volatile memory; processing the input data through multiple sub-operations in each layer, updating the progress tracking variables upon completion of each sub-operation, and writing the output of the sub-operation and the corresponding progress tracking variables into non-volatile memory; if a processing interruption occurs during the processing of input data through multiple sub-operations in each layer, determining the completed sub-operations based on the progress tracking variables in non-volatile memory, retrieving the input data and model parameters required for the remaining sub-operations, and completing the remaining sub-operations. This eliminates the need for application partitioning inference, thus avoiding energy estimation.

Description

Technical Field

[0001] This application relates to the field of neural network inference methods, and more particularly to intermittent deep neural network inference methods for energy harvesting Internet of Things. Background Technology

[0002] In the field of smart IoT, IoT devices typically integrate sensors, microcontrollers, and wireless communication functions, using various types of batteries as power sources. Battery technology faces several challenges: First, the limited lifespan of batteries necessitates regular replacement, which is difficult and risky in some situations, such as heart pacemakers. Second, the environmental pollution caused by the use of large numbers of batteries is a significant issue; many micro-IoT devices, once deployed, have batteries that are almost impossible to recycle, causing severe environmental pollution. With the widespread application of IoT devices, energy harvesting technology has emerged as an alternative energy solution. IoT devices powered by energy harvesting technology are collectively referred to as energy-harvesting IoT devices. The core component of these energy-harvesting IoT devices is the energy harvester, responsible for capturing energy from the environment, such as solar, wind, thermal, or kinetic energy. In this way, energy-harvesting IoT devices can utilize the natural resources of their surroundings to achieve self-powering, thereby reducing reliance on traditional batteries, lowering maintenance costs, and mitigating environmental impact. Energy-harvesting IoT devices have been developed for various devices, which are becoming increasingly intelligent, with the future trend being the operation of complex deep neural network calculations on edge-restricted devices. Energy harvesting IoT devices currently face a significant challenge: the energy harvested from the surrounding environment is uncertain due to environmental variability, resulting in typically small and unstable energy output. This instability in energy sources frequently leads to program interruptions and loss of computational context during task execution due to power failures, requiring devices to restart calculations from scratch after power restoration.

[0003] Checkpointing is a widely used technique in fault-tolerant computing. When program execution reaches a checkpoint, the system state is copied from volatile memory to non-volatile memory, allowing the program to resume execution from that point. This technique is widely used in intermittent computing. Checkpointing is programmer-friendly because it typically doesn't require the programmer to specify the data to be saved. This approach provides an effective strategy for addressing computational interruptions caused by power instability, especially for energy harvesting IoT devices, ensuring continuous program operation even under frequent power outages. Task-based breakpoint save-and-restore techniques rewrite the original application as a series of small, idempotent task sequences. Task execution is atomic; it either completes or is aborted. Due to the idempotency of tasks, the key to these techniques is that task outputs are not overwritten.

[0004] Traditional checkpoint-based intermittent computing techniques fail to effectively reinitialize and wake up the accelerator, potentially causing the CPU to miss timely accelerator completion signals and hindering progress. Furthermore, static checkpoint settings may incur unnecessary overhead or fail to guarantee operational efficiency due to a lack of adaptability, while dynamic evaluation and optimization processes themselves can introduce significant time and energy overhead, particularly in energy-constrained environments. Task-based intermittent computing techniques heavily rely on accurate energy estimation of application partitions. However, deep neural networks have multiple computational layers of varying complexity, making proper partitioning difficult. Moreover, estimating the energy cost of a task and the available energy budget is not straightforward. Energy miscalculations can lead to costly task re-executions, impacting inference response time, increasing the programmer's workload, and complicating task partitioning and management, thus increasing implementation and maintenance difficulties. Summary of the Invention

[0005] The main objective of this application is to provide an intermittent deep neural network inference method for energy harvesting Internet of Things, which aims to address the problem of accurate energy estimation based on task-based intermittent computing techniques that rely on application partitioning.

[0006] To achieve the above objectives, this application provides an intermittent deep neural network inference method for energy harvesting Internet of Things (IoT), comprising: obtaining input data, model parameters, and progress tracking variables of each layer of a deep neural network model from non-volatile memory; processing the input data through multiple sub-operations in each layer, updating the progress tracking variables upon completion of each sub-operation, and writing the output of the sub-operation and the corresponding progress tracking variables into the non-volatile memory; wherein, the deep neural network model includes convolutional layers, pooling layers, and fully connected layers, the processing of input data by the convolutional layers and fully connected layers is based on CPU computation and / or hardware accelerator computation, and the processing of input data by the pooling layers is based on CPU computation; if a processing interruption occurs during the processing of input data through multiple sub-operations in each layer, the sub-operations that have been completed at the time of the interruption are determined according to the progress tracking variables in the non-volatile memory, and the input data and model parameters required for the remaining sub-operations are obtained to complete the remaining sub-operations.

[0007] Optionally, the progress tracking variables include a first progress tracking variable and a second progress tracking variable.

[0008] Optionally, when the convolutional layer processes the input data based on CPU computation, the method includes: obtaining the input data and model parameters of the convolutional layer, as well as a first progress tracking variable and a second progress tracking variable, from non-volatile memory; extracting features sequentially through multiple sub-operations, updating the first progress tracking variable upon completion of each sub-operation, and writing the output of the sub-operation and the corresponding first progress variable into non-volatile memory; obtaining the output feature map upon completion of all sub-operations, updating the second progress tracking variable, and writing the second progress tracking variable into non-volatile memory.

[0009] Optionally, when the convolutional layer processes the input data based on hardware accelerator computation, the method includes: obtaining the input data and model parameters of the convolutional layer, as well as the first and second progress tracking variables, from non-volatile memory via direct memory access; extracting features sequentially through multiple sub-operations, updating the first progress variable after each sub-operation, and writing the sub-operation results and the corresponding first progress variables into non-volatile memory; obtaining the output feature map when all sub-operations are completed, updating the second progress tracking variable, and writing the output feature map and the updated second progress tracking variable into non-volatile memory via direct memory access.

[0010] Optionally, the suboperation includes a multiplication operation and a corresponding accumulation operation.

[0011] Optionally, when the pooling layer processes the input data based on CPU computation, the method includes: retrieving the input data, model parameters, and second progress tracking variables of the pooling layer from non-volatile memory; sequentially performing multiple sub-operations on the input data, generating an element at each position of the output feature map and outputting it; determining the position of the currently completed sub-operation based on the quotient and remainder of the product of the current output feature map size; updating the second progress tracking variable when each sub-operation is completed, and writing the output feature map of the current sub-operation and the corresponding second progress tracking variable into non-volatile memory; wherein, the sub-operation includes the pooling operation at each position.

[0012] Optionally, when the fully connected layer processes the input data based on CPU computation, the method includes: obtaining the input data and model parameters of the fully connected layer, as well as the second progress tracking variable, from non-volatile memory; traversing each output node, performing sub-operations on its corresponding input data for each of all input nodes, updating the second progress tracking variable upon completion of the sub-operation, and writing the output result of the sub-operation and its corresponding second progress tracking variable into non-volatile memory; the sub-operations include weighted sum operation and application of activation function, or weighted sum operation and setting to zero; wherein, when the value of the weighted sum is greater than zero, the output node applies the activation function, and when the value of the weighted sum is less than zero, the output node is set to zero.

[0013] Optionally, when the fully connected layer processes the input data based on hardware accelerator computation, the method includes: obtaining the input data and model parameters of the fully connected layer in non-volatile memory, as well as the second progress tracking variable, through direct memory access; performing sub-operations for each output channel; after obtaining the sub-operation results for each output channel, updating the second progress tracking variable, and writing the sub-operation results and the corresponding second progress tracking variable into non-volatile memory; wherein, the sub-operation includes a multiply-accumulate operation, and the result of the multiply-accumulate operation is added to the bias value, and an activation function is applied.

[0014] Optionally, when the deep neural network inference process is executed based on CPU computation, the input data, model parameters, and progress tracking variables are all passed from non-volatile memory to the corresponding layer of the deep neural network model through pointers.

[0015] Compared with the prior art, the beneficial effects of this application are as follows:

[0016] This invention presents an intermittent deep neural network inference method for the Internet of Things (IoT) of energy harvesting. Leveraging the unidirectional nature of deep neural network inference and the sequential nature of sub-operations, it progressively advances the inference process without requiring application partitioning for inference, thus avoiding energy estimation. It calculates the progress of sub-operation completions and stores the progress and corresponding sub-operation results in non-volatile memory. The CPU and accelerator interact through this non-volatile memory, receiving accelerator completion signals and progress updates. Only sub-operation results are saved, eliminating the need to store numerous intermediate states, minimizing runtime costs and avoiding excessive space consumption. Furthermore, it effectively captures and saves inference progress to prevent loss due to power interruptions. Upon interruption recovery, the interruption location is determined based on the progress, and the corresponding data for unexecuted sub-operations is retrieved and transmitted to the CPU or hardware accelerator to resume the interrupted inference process, improving the speed and accuracy of data recovery. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the intermittent deep neural network inference method for the Internet of Things for energy harvesting proposed in this application;

[0018] Figure 2 This is a comparison chart of execution times for Embodiment 1 of the intermittent deep neural network inference method for the Internet of Things for energy harvesting in this application;

[0019] Figure 3 This is a comparison chart of energy consumption in Embodiment 1 of the intermittent deep neural network inference method for energy harvesting Internet of Things in this application;

[0020] Figure 4 This is a graph showing the inference throughput variation under different capacitors in Embodiment 1 of the intermittent deep neural network inference method for energy harvesting Internet of Things in this application.

[0021] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] A monocrystalline solar panel with a rated output of 5V 200mA is used as the power supply unit to simulate the solar energy harvesting process. The energy harvesting unit consists of a boost converter, a series of energy buffers, and a power control switch. In an intermittent energy harvesting computing system, considering that the output power of the solar panel is affected by various factors such as light intensity, weather conditions, and angle, power fluctuations or power instability may occur. Capacitors can store excess energy when the light conditions are good, and then release the energy when the light is insufficient or there is no light, ensuring that the MCU receives a continuous and stable power supply. Capacitors can also act as energy buffers, smoothing out instantaneous fluctuations in the solar panel output and preventing power interruptions caused by brief cloud cover or changes in light intensity. Under such a power configuration, the execution of a single complete inference process often cannot be completed in one go, resulting in frequent power interruptions and recovery. To address the above issues, this invention provides an intermittent deep neural network inference method for energy harvesting IoT, such as... Figure 1 As shown, the specific steps include:

[0024] Before performing inference, a deep neural network model needs to be loaded using the API to execute the inference. The specific inference method is as follows.

[0025] Step S1: Obtain the input data, model parameters, and progress tracking variables of each layer of the deep neural network model from non-volatile memory; process the input data through multiple sub-operations in each layer, update the progress tracking variables when each sub-operation is completed, and write the output of the sub-operation and the corresponding progress tracking variables into non-volatile memory; wherein, the deep neural network model includes convolutional layers, pooling layers, and fully connected layers. The processing of input data by the convolutional layers and fully connected layers is based on CPU computation and / or hardware accelerator computation, and the processing of input data by the pooling layers is based on CPU computation.

[0026] For example, the input to the convolutional layer can be an image, and the deep neural network model inference method of this embodiment can be used for image recognition, etc.

[0027] In this embodiment, the inference of the deep neural network model can be performed based on CPU computation, or it can be performed using a combination of CPU computation and hardware accelerator computation. Specifically, when the hardware accelerator is enabled, the CPU enters a sleep state and is awakened after the hardware accelerator operation is completed. Furthermore, the update of the progress tracking variable can be implemented using a counter. Initially, the progress tracking variable is set to 0, and it is updated once after each sub-operation is completed, i.e., the progress tracking variable is incremented, and the operation result of that sub-operation is saved. Thus, if the inference process is interrupted, the interruption point and the corresponding operation result of the completed sub-operation can be found based on the progress tracking variable, and the calculation can continue from the correct position. Utilizing the unidirectional nature of deep neural network inference and the sequential nature of sub-operations, the inference process is progressively advanced, saving only the result of each sub-operation and eliminating the need to save a large number of intermediate states.

[0028] Furthermore, when the deep neural network inference process is executed based on CPU computation, the input data, model parameters, and progress tracking variables are all passed from non-volatile memory to the corresponding layer of the deep neural network model through pointers. The pointers contain the receiving addresses of the input data, model parameters, and progress tracking variables, and point to the corresponding processing layer.

[0029] Progress tracking variables include a first progress tracking variable and a second progress tracking variable. When the processing of input data in the convolutional layer is based on CPU computation, the inference method of the convolutional layer specifically includes the following steps:

[0030] The input data and model parameters of the convolutional layer, as well as the first and second progress tracking variables, are obtained from non-volatile memory using pointers. Features are extracted sequentially through multiple sub-operations. Upon completion of each sub-operation, the first progress tracking variable is updated, and the output of the sub-operation and the corresponding first progress variable are written to non-volatile memory. When all sub-operations are completed, i.e., when the convolution operation is completed, the feature map is output, the second progress tracking variable is updated, and the second progress tracking variable is written to non-volatile memory.

[0031] Specifically, the input data for a convolutional layer can be the original input data or the output data from the previous layer. Its model parameters include weights (convolutional kernel) and biases. Sub-operations include a multiplication operation and a corresponding accumulation operation, that is, the convolutional kernel performs element-wise multiplication on each element within the region covered by the input data and the corresponding accumulation operation.

[0032] When convolutional layers process input data using hardware accelerators, the methods include:

[0033] The input data and model parameters of the convolutional layer, as well as the first and second progress tracking variables, are obtained from non-volatile memory via direct memory access. Features are extracted sequentially through multiple sub-operations. After each sub-operation, the first progress variable is updated, and the sub-operation result and the corresponding first progress variable are written to non-volatile memory. Upon completion of all sub-operations, the output feature map is obtained, the second progress tracking variable is updated, and the output feature map and the updated second progress tracking variable are written to non-volatile memory via direct memory access. Each sub-operation includes a multiplication operation and a corresponding accumulation operation.

[0034] After writing the output feature map and the updated second progress tracking variable into non-volatile memory via direct memory access, the CPU needs to be woken up to execute the pooling layer inference. The wake-up method can be that when the current second progress tracking variable reaches a preset value, the CPU receives an accelerator completion signal and makes progress, at which point the CPU begins executing the pooling layer inference. Specific inference methods include:

[0035] The input data, model parameters, and second progress tracking variable of the pooling layer are obtained from non-volatile memory. Multiple sub-operations are performed on the input data in sequence, generating elements at their corresponding output feature map positions and outputting them. The output element positions of the completed sub-operations are determined by taking the quotient and remainder of the product of the current output feature map sizes. When each sub-operation is completed, i.e., each time the position changes, the second progress tracking variable is updated, and the output feature map of the current sub-operation and the corresponding second progress tracking variable are written to non-volatile memory. The sub-operations include pooling operations at each position.

[0036] Specifically, the input data for the pooling layer can be the output of the convolutional layer. The product of the dimensions of the output feature map is the product of the height H, the width W, and the number of channels CH.

[0037] When the fully connected layer processes input data based on CPU computation, the method includes: retrieving the input data and model parameters of the fully connected layer, as well as the second progress tracking variable, from non-volatile memory; traversing each output node, performing sub-operations on its corresponding input data for each input node, updating the second progress tracking variable upon completion of the sub-operation, and writing the output result of the sub-operation and its corresponding second progress tracking variable into non-volatile memory; the sub-operations include weighted sum operation and applying an activation function, or weighted sum operation and setting it to zero; wherein, when the value of the weighted sum is greater than zero, the activation function is applied to the output node, and when the value of the weighted sum is less than zero, the output node is set to zero.

[0038] The input data for the fully connected layer can be the output of the pooling layer, and the model parameters include weights.

[0039] When a fully connected layer processes input data based on hardware accelerator computation, the methods include:

[0040] The input data and model parameters of the fully connected layer, as well as the second progress tracking variable, are obtained from non-volatile memory through direct memory access. For each output channel, a sub-operation is performed. After obtaining the sub-operation result for each output channel, the second progress tracking variable is updated, and the sub-operation result and the corresponding second progress tracking variable are written into non-volatile memory. The sub-operation includes multiplication and accumulation operations. The result of the multiplication and accumulation operations is added to the bias value, and an activation function is applied.

[0041] Step S2: If a processing interruption occurs during the processing of input data through multiple sub-operations in each layer, for example, if a power interruption occurs during step 1 causing inference to stop, then after the power is restored, the sub-operations that were completed when the interruption occurred are determined based on the progress tracking variables in non-volatile memory, and the input data and model parameters required for the remaining sub-operations are obtained to complete the remaining sub-operations.

[0042] In this embodiment, the sub-operations that have been completed when the interruption occurs are determined based on the progress tracking variables. Only the input data and model parameters required for the unexecuted sub-operations are obtained, which can reduce latency and energy consumption. It can operate normally even under relatively small energy buffer conditions. Based on the progress tracking variables, it can be ensured that recovery is performed from the correct position.

[0043] Example 1

[0044] The inference method of this invention is used to perform inference on the CNN model for image handwritten digit recognition. The parameters of the CNN model are shown in Table 1.

[0045] Table 2 Dataset Parameters

[0046]

[0047] Operating overhead and inference throughput were set as evaluation metrics based on workload and capacitor size. Operating overhead was determined by the execution time and energy consumption required to complete one full inference under stable continuous power conditions. Energy consumption was measured using Texas Instruments' EnergyTrace tool. Pure software deep neural networks running on a CPU and deep neural networks running on a hardware accelerator were run separately. Inference throughput under intermittent power conditions was measured by the number of successful end-to-end inferences for all test methods and configurations within the same experimental cycle.

[0048] Regarding runtime overhead, the energy consumption and execution time of running deep neural networks were compared under four scenarios: Implementation 1, CPU-based (i.e., software-level) inference implementation; Implementation 2, CPU-based inference of pooling layers and hardware accelerator-based inference of convolutional and fully connected layers; the above two do not handle intermittent computation issues and serve as the baseline comparison experiments, where the state is lost during power interruption and the entire application restarts upon power restoration; Implementation 3, CPU-based intermittent complete deep neural network model inference implementation; Implementation 4, CPU-based intermittent inference of pooling layers and hardware accelerator-based intermittent inference of convolutional and fully connected layers.

[0049] Figure 2 and Figure 3 The paper presents the execution time and energy consumption of four inference implementations of a CNN model under continuous power to complete a single end-to-end inference operation. The first two implementations serve as the baseline experimental control group for this study, and the final results are normalized based on implementation 4. A single inference operation in the handwritten digit recognition application requires 1486 ms of execution time and consumes 11.03 mJ of energy. It can be seen that energy consumption is directly related to inference execution time, thus showing a similar trend. Comparing implementation 1 and implementation 2, hardware acceleration provides 2 to 8 times faster execution time and 3 to 10 times energy savings. Comparing implementation 1 and implementation 3, after considering power interruptions and employing the inference method of this invention, its execution time and energy consumption are comparable to the pure software implementation, indicating that this intermittent processing does not introduce excessive runtime overhead.

[0050] Under intermittent power conditions, the inference throughput of all implementations and deep neural network applications varies across different capacitance levels. Higher capacitance (i.e., a larger energy budget) allows for longer execution times during power outages, resulting in higher throughput. Larger and more complex deep neural network models require more time and energy to execute, leading to lower throughput. The number of executions completing a full inference within 5 minutes was repeatedly counted and averaged.

[0051] To examine the system's performance under different energy conditions, experiments were conducted using energy buffer capacitors of varying capacities and under different energy budgets, such as... Figure 4As shown in the figure, the inference throughput under different capacitors can be observed. With a 2200 microfarad capacitor, the handwritten digit recognition model executes 24 times using Implementation 3 within a given time, and 92 times using Implementation 4. The handwritten digit recognition model requires several power cycles to complete one inference operation. Implementations 1 and 2 lack intermittent support, therefore exhibiting zero throughput when the available energy is insufficient to complete a single end-to-end inference operation within a power cycle. Ultimately, it can be seen that the intermittent deep neural network model inference system of this invention can still ensure inference progress even when a full computation cannot be supplied in a single power cycle. In an environment without a hardware accelerator, CPU-based inference computation (Experiment 3) can also advance the inference progress, while the intermittent deep neural network model inference system that can use a hardware accelerator (Experiment 4) is even more efficient.

[0052] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An intermittent deep neural network inference method for energy harvesting Internet of Things, characterized in that, include: Retrieve input data, model parameters, and progress tracking variables for each layer of the deep neural network model from non-volatile memory; The input data is processed through multiple sub-operations in each layer. When each sub-operation is completed, the progress tracking variable is updated, and the output of the sub-operation and the corresponding progress tracking variable are written into non-volatile memory. The deep neural network model includes convolutional layers, pooling layers, and fully connected layers. The convolutional layers and fully connected layers process the input data based on CPU operations and / or hardware accelerator operations, and the pooling layers process the input data based on CPU operations. If a processing interruption occurs during the processing of input data through multiple sub-operations in each layer, the sub-operations that have been completed at the time of the interruption are determined based on the progress tracking variables in the non-volatile memory, and the input data and model parameters required for the remaining sub-operations are obtained to complete the remaining sub-operations. The progress tracking variables include a first progress tracking variable and a second progress tracking variable; The progress tracking variable is updated using a counter. It is updated once after each sub-operation is completed, and the result of the sub-operation is saved. When the inference process is interrupted, the interruption position and the corresponding result of the completed sub-operation are found based on the progress tracking variable, and the calculation continues from the correct position. The sub-operation includes a multiplication operation and a corresponding accumulation operation; When the deep neural network inference process is executed based on CPU computation, the input data, model parameters, and progress tracking variables are all passed from non-volatile memory to the corresponding layer of the deep neural network model through pointers.

2. The intermittent deep neural network inference method for energy harvesting Internet of Things according to claim 1, characterized in that, When the processing of input data by the convolutional layer is based on CPU computation, the method includes: Obtain the input data and model parameters of the convolutional layer, as well as the first and second progress tracking variables, from non-volatile memory; Features are extracted sequentially through multiple sub-operations. When each sub-operation is completed, the first progress tracking variable is updated, and the output of the sub-operation and the corresponding first progress variable are written into non-volatile memory. When all sub-operations are completed, the output feature map is obtained, the second progress tracking variable is updated, and the second progress tracking variable is written into non-volatile memory.

3. The intermittent deep neural network inference method for energy harvesting Internet of Things according to claim 1, characterized in that, When the convolutional layer processes input data based on hardware accelerator computation, the method includes: The input data and model parameters of the convolutional layer in non-volatile memory, as well as the first and second progress tracking variables, are obtained through direct memory access. Features are extracted sequentially through multiple sub-operations. After each sub-operation is completed, the first progress variable is updated, and the sub-operation results and the corresponding first progress variable are written into non-volatile memory. When all sub-operations are completed, an output feature map is obtained, the second progress tracking variable is updated, and the output feature map and the updated second progress tracking variable are written into non-volatile memory via direct memory access.

4. The intermittent deep neural network inference method for energy harvesting Internet of Things according to claim 2 or 3, characterized in that, When the pooling layer processes input data based on CPU computation, the method includes: Retrieve the input data, model parameters, and second progress tracking variables of the pooling layer from non-volatile memory; Multiple sub-operations are performed sequentially on the input data, generating an element at each position in the output feature map and outputting it; the position of the currently completed sub-operation is determined by taking the quotient and remainder of the product of the current output feature map sizes. Upon completion of each sub-operation, the second progress tracking variable is updated, and the output feature map of the current sub-operation and the corresponding second progress tracking variable are written into non-volatile memory. The sub-operations include pooling operations at each location.

5. The intermittent deep neural network inference method for energy harvesting Internet of Things according to claim 4, characterized in that, When the fully connected layer processes input data based on CPU computation, the method includes: Retrieve the input data and model parameters of the fully connected layer, as well as the second progress tracking variables, from non-volatile memory; Iterate through each output node, perform sub-operations on the input data corresponding to each input node, update the second progress tracking variable when the sub-operation is completed, and write the output result of the sub-operation and its corresponding second progress tracking variable into non-volatile memory. The sub-operations include weighted summation and application of an activation function, or weighted summation and setting to zero; wherein, the activation function is applied to the output node when the value of the weighted sum is greater than zero, and the output node is set to zero when the value of the weighted sum is less than zero.

6. The intermittent deep neural network inference method for energy harvesting Internet of Things according to claim 4, characterized in that, When the fully connected layer processes input data based on hardware accelerator computation, the method includes: The input data and model parameters of the fully connected layer in non-volatile memory, as well as the second progress tracking variables, are obtained through direct memory access. For each output channel, perform a sub-operation. After obtaining the sub-operation result for each output channel, update the second progress tracking variable and write the sub-operation result and the corresponding second progress tracking variable into non-volatile memory. The sub-operation includes a multiply-accumulate operation, and the result of the multiply-accumulate operation is added to the bias value and an activation function is applied.

Images