Convolution accelerator, control method of convolution operation, and micro control circuit

Abstract

This invention discloses a convolution accelerator, a control method for convolution operations, and a microcontroller circuit. The convolution accelerator is used to perform multi-layer convolution operations on data stored in a storage circuit, and includes multiple convolution calculation circuits, a detection circuit, and a control circuit. Each convolution calculation circuit performs a convolution operation when activated. The detection circuit detects an access state of the storage circuit within a fixed time period to generate a detection signal. The control circuit dynamically adjusts the number of convolution calculation circuits activated for each layer of convolution operations based on the detection signal.

Description

Technical Field

[0001] This invention relates to a convolution accelerator, and more particularly to a convolution accelerator that dynamically adjusts the number of convolution calculation circuits. Background Technology

[0002] In recent years, deep neural networks have been widely used in various fields, with convolutional neural networks (CNNs) being the mainstream. As model complexity increases, CNN hardware architectures also become more complex. Taking image recognition as an example, suppose a CNN hardware architecture has a frame rate of 120Hz. If the user doesn't need such a high frame rate, or if the camera only supports a frame rate of 60Hz, a large number of circuits within the CNN hardware architecture will be idle. This reduces hardware utilization and increases power consumption. Summary of the Invention

[0003] This invention provides a convolution accelerator for performing multi-layer convolution operations on data stored in a storage circuit, and includes multiple convolution calculation circuits, a detection circuit, and a control circuit. Each convolution calculation circuit performs a convolution operation when activated. The detection circuit detects an access state of the storage circuit within a fixed time period to generate a detection signal. The control circuit dynamically adjusts the number of convolution calculation circuits activated for each layer of convolution operations based on the detection signal.

[0004] The present invention also provides a microcontroller circuit, including a sensing circuit, a storage circuit, a convolution accelerator, and a processing circuit. The sensing circuit detects an external image to generate input data. The storage circuit stores the input data. The convolution accelerator performs multi-layer convolution operations on the input data and includes multiple convolution calculation circuits, a detection circuit, and a control circuit. Each convolution calculation circuit performs a convolution operation when activated. The detection circuit detects an access state of the storage circuit within a fixed time period to generate a detection signal. The control circuit dynamically adjusts the number of convolution calculation circuits to be activated for each layer of convolution operation based on the detection signal. The processing circuit triggers the convolution accelerator and identifies the external image based on the operation results of the convolution calculation circuits.

[0005] The present invention further provides a control method for convolution operations, comprising: storing image data in a storage circuit; monitoring an access state of the storage circuit within a fixed time period; determining, based on the access state, the number of convolutional computing circuits to be activated for each layer of convolutional operations in a first processing stage; detecting a first operation result generated by the first processing stage; determining, based on the data volume of the first operation result and the access state, the number of convolutional computing circuits to be activated for each layer of convolutional operations in a second processing stage; detecting a second operation result generated by the second processing stage; detecting a frame rate to determine a processing time; calculating a total time for the first and second processing stages; calculating a difference between the processing time and the total time; and determining, based on the data volume of the second operation result and the difference, the number of convolutional computing circuits to be activated for each layer of convolutional operations in a third processing stage.

[0006] The control method for convolution operations of the present invention can be implemented via the microcontroller circuit and convolution accelerator of the present invention, which can be hardware or firmware capable of performing specific functions, or can be implemented by recording program code in a recording medium and combined with specific hardware. When the program code is loaded and executed by an electronic device, processor, computer or machine, the electronic device, processor, computer or machine becomes the microcontroller circuit and convolution accelerator used to implement the present invention. Attached Figure Description

[0007] Figure 1 This is a schematic diagram of the microcontroller circuit of the present invention.

[0008] Figure 2 This is a schematic diagram of the convolution accelerator of the present invention.

[0009] Figure 3 This invention relates to a method for controlling convolution operations.

[0010] Symbol Explanation

[0011] 100: Microcontroller circuit

[0012] 110: Sensing Circuit

[0013] 120, 133: Storage circuit

[0014] 130: Convolution Accelerator

[0015] 131: Control Circuit

[0016] 132: Detection Circuit

[0017] 140: Processing Circuit

[0018] IN: Input data

[0019] CV_1~CV_4: Convolution Calculation Circuits

[0020] SD: Detection signal

[0021] SO1~SO4: Enable signal

[0022] 211, 221, 231, 241: Input buffers

[0023] 212, 222, 232, 242: Processing engine circuits

[0024] 213, 223, 233, 243: Accumulators

[0025] 214, 224, 234, 244: Output buffers

[0026] DI_1~DI_4: Data

[0027] DW_1~DW_4: Weight values

[0028] S311~S313: Steps Detailed Implementation

[0029] To make the objectives, features, and advantages of this invention more apparent and understandable, embodiments are provided below in conjunction with the accompanying drawings for detailed description. This specification provides different embodiments to illustrate the technical features of different implementations of the invention. The configuration of the elements in the embodiments is for illustrative purposes only and is not intended to limit the invention. Furthermore, the repetition of some reference numerals in the embodiments is for simplification and does not imply any correlation between different embodiments.

[0030] Figure 1 This is a schematic diagram of the microcontroller circuit of the present invention. As shown, the microcontroller circuit 100 includes a sensing circuit 110, a storage circuit 120, a convolution accelerator 130, and a processing circuit 140. The sensing circuit 110 detects an external state to generate input data IN. The present invention does not limit the type of sensing circuit 110. In one possible embodiment, the sensing circuit 110 is an image sensor for detecting an external image. In this example, the input data IN is frame data. In another possible embodiment, the sensing circuit 110 is a sound sensor for detecting an external sound.

[0031] Storage circuit 120 is used to store input data IN. The present invention is not limited to the architecture of storage circuit 120. In one possible embodiment, storage circuit 120 has a memory and a memory controller. The memory controller accesses the memory according to an external instruction. In some embodiments, storage circuit 120 includes a volatile memory circuit, such as static random-access memory (SRAM).

[0032] The convolution accelerator 130 accesses the storage circuit 120 for performing multi-layer convolution operations on the input data IN. In this embodiment, the convolution accelerator 130 includes convolution calculation circuits CV_1 to CV_4. Taking convolution calculation circuit CV_1 as an example, when convolution calculation circuit CV_1 is turned on, it performs a convolution operation. The present invention does not limit the number of convolution calculation circuits. In other embodiments, the convolution accelerator 130 may have more or fewer convolution calculation circuits.

[0033] In some embodiments, the convolution accelerator 130 further includes a control circuit 131 and a detection circuit 132. The detection circuit 132 detects the operating state of the microcontroller circuit 100 to generate a detection signal SD. In one possible embodiment, the detection circuit 132 is a bus monitor that detects an access state of the storage circuit 120 over a fixed period of time, such as detecting the idle time and / or free space of the storage circuit 120.

[0034] For example, the bus monitor collects signals from other circuits (circuits other than convolution accelerator 130) accessing memory circuit 120, such as a read / write signal, a ready signal, and a response signal. Based on the collected information, the bus monitor calculates the proportion of time memory circuit 120 is accessed over a fixed period of time. For example, in 1000 cycles, memory circuit 120 is accessed by other circuits for 30% of the cycles. In this example, memory circuit 120 has 70% of idle cycles.

[0035] Based on the detection signal SD, the control circuit 131 dynamically adjusts the number of convolutional computing circuits to be activated for each layer of convolutional operations in different processing stages. In each processing stage, the activated convolutional computing circuits may perform convolutional operations on multiple layers. For example, in the first processing stage, before performing convolutional operations on each layer, the control circuit 131 first determines, based on the detection signal SD, how much of the available free time and space in the storage circuit 120 can satisfy the needs of the convolutional computing circuits, and then, based on the determination result, activates the maximum number of convolutional computing circuits to perform the convolutional operation. The number of activated convolutional computing circuits in the first processing stage is called a first quantity.

[0036] For example, suppose control circuit 131 determines from the detection signal SD that storage circuit 120 will not be accessed by other circuits for 70% of 1000 cycles. In this case, control circuit 131 may activate convolution calculation circuits CV_1 to CV_3. However, if control circuit 131 activates convolution calculation circuits CV_1 to CV_4, since the bandwidth of storage circuit 120 can only satisfy three convolution calculation circuits (such as CV_1 to CV_3), one convolution calculation circuit (such as CV_4), although activated, cannot access storage circuit 120. The idle convolution calculation circuit will cause excessive power loss. However, if the bandwidth of storage circuit 120 can satisfy the operation of four convolution calculation circuits, control circuit 131 may activate all convolution calculation circuits (such as CV_1 to CV_4).

[0037] In other embodiments, the control circuit 131 writes the operation result of each convolution calculation circuit into the storage circuit 120. The detection circuit 132 detects the amount of data in the operation result of the storage circuit 120. In a second processing stage, the control circuit 131 determines the number of convolution calculation circuits to be activated based on the detection result of the detection circuit 132. At this time, the number of activated convolution calculation circuits is called a second quantity.

[0038] For example, if the amount of data resulting from the first processing stage is below a threshold, the control circuit 131 activates a smaller number of convolution calculation circuits, such as activating convolution calculation circuits CV_1 and CV_2, and deactivating convolution calculation circuits CV_3 and CV_4. In this case, if the control circuit 131 considers the available time and space for the storage circuit 120 to access the data, it can configure the most efficient use of the convolution calculation circuits. For example, even if the amount of data resulting from the processing is below a threshold, if the bandwidth of the storage circuit 120 is insufficient to support convolution calculation circuits CV_1 and CV_2, the control circuit 131 may only activate convolution calculation circuit CV_1.

[0039] In some embodiments, the detection circuit 132 further detects a frame rate. In a third processing stage, the control circuit 131 determines the frames per second (FPS) based on the detection result of the detection circuit 132, and determines how much processing time remains based on the FPS. In this example, the control circuit 131 activates the minimum number of convolutional computation circuits to perform convolution operations on the computation results generated in the second processing stage based on the remaining processing time, in order to achieve a low power consumption target and improve the overall hardware utilization. The number of convolutional computation circuits activated in the third processing stage is called a third quantity. In one possible embodiment, the first quantity is greater than the second quantity, and the second quantity is greater than the third quantity. In other embodiments, the first quantity may be equal to the second quantity and greater than the third quantity.

[0040] This invention does not limit how the control circuit 131 activates the convolution calculation circuits CV_1 to CV_4. In one possible embodiment, the control circuit 131 uses activation signals SO1 to SO4 to control the convolution calculation circuits CV_1 to CV_4. Taking the convolution calculation circuit CV_1 as an example, when the activation signal SO1 is enabled, the convolution calculation circuit CV_1 is activated. When the activation signal SO1 is disabled, the convolution calculation circuit CV_1 is not activated. In another possible embodiment, the control circuit 131 sends an activation command (not shown) to the convolution calculation circuits CV_1 to CV_4. In this example, the activation command may have four bits, each corresponding to a convolution calculation circuit. When the first bit is a first value (e.g., value 1), the corresponding convolution calculation circuit (e.g., CV_1) starts operating. When the first bit is a second value (e.g., value 0), the corresponding convolution calculation circuit (e.g., CV_1) stops operating. In other embodiments, the control circuit 131 controls the clock signals of the convolution calculation circuits CV_1 to CV_4. Taking the convolution calculation circuit CV_1 as an example, when the control circuit 131 stops providing a clock signal to the convolution calculation circuit CV_1, the convolution calculation circuit CV_1 does not operate. When the control circuit 131 provides a clock signal to the convolution calculation circuit CV_1, the convolution calculation circuit CV_1 starts operating.

[0041] In some embodiments, the control circuit 131 uses a look-up table (LUT) or a machine learning model to adjust the number of convolutional computation circuits to be activated for each layer of convolutional operations. For example, the control circuit 131 has a storage circuit 133 for storing a look-up table or a machine learning model.

[0042] In one possible embodiment, the storage circuit 133 stores a lookup table. The lookup table records multiple scenarios. The control circuit 131, based on the detection signal SD, the calculation results of the first processing stage, the calculation results of the second processing stage, the remaining time, and the frame rate, retrieves the corresponding quantity value for the scenario from the lookup table, and then determines the number of convolution calculation circuits to be activated based on the quantity value. In this example, each scenario corresponds to a detection signal SD, a quantity of first calculation results (i.e., the amount of computation required for the second processing stage), a quantity of second calculation results (i.e., the amount of computation required for the third processing stage), a remaining time value, and a frame rate.

[0043] In one possible embodiment, storage circuit 133 stores a machine learning model. During a training phase before performing convolution operations on the input data IN, control circuit 131 inputs training data to the machine learning model to train it to predict the appropriate number of convolutional computation circuits to be activated before each layer's convolutional operation. In this example, the training data includes multiple access states (such as the output of detection circuit 132), multiple computation results (such as the remaining computation after convolution operations on different frame data), and multiple time values ​​(such as remaining processing time). Control circuit 131 uses the pre-trained machine learning model to determine the number of convolutional computation circuits to be activated for each layer's convolutional operation. This invention does not limit the type of machine learning model. In one possible embodiment, the machine learning model is a recurrent neural network (RNN), such as a Long Short-Term Memory Network (LSTM) model or a gated recurrent unit (GRU) neural network model.

[0044] The processing circuit 140 is used to trigger the convolution accelerator 130 and perform a recognition operation based on the operation results of the convolution calculation circuits CV_1 to CV_4, such as identifying whether an external image matches a target image. In some embodiments, the processing circuit 140 performs a fully connected layer operation on the operation results of the convolution calculation circuits CV_1 to CV_4.

[0045] Figure 2This is a schematic diagram of the convolution accelerator 130 of the present invention. The convolution accelerator 130 includes convolution calculation circuits CV_1 to CV_4. Convolution calculation circuit CV_1 includes an input buffer 211, a processing engine circuit 212, an accumulator 213, and an output buffer 214. Convolution calculation circuit CV_2 includes an input buffer 221, a processing engine circuit 222, an accumulator 223, and an output buffer 224. Convolution calculation circuit CV_3 includes an input buffer 231, a processing engine circuit 232, an accumulator 233, and an output buffer 234. Convolution calculation circuit CV_4 includes an input buffer 241, a processing engine circuit 242, an accumulator 243, and an output buffer 244. Since the operation of convolution calculation circuits CV_1 to CV_4 is similar, the operation of convolution calculation circuit CV_1 will be described below as an example.

[0046] Input buffer 211 stores a data DI_1 and multiple weight values ​​DW_1. In one possible embodiment, the data DI_1 and the weight values ​​DW_1 are provided by storage circuitry 120. The data DI_1 is a portion of the input data IN. In some embodiments, the data DI_1 is an input feature map (IFM). In this example, the data DI_1 may be a 4x4 matrix. In other embodiments, the weight values ​​DW_1 are a 3x3 matrix.

[0047] The processing engine circuit 212 operates on the data DI_1 and the weight value DW_1 to generate multiple output results. In one possible embodiment, the processing engine circuit 212 performs a convolution operation on the data DI_1 and the weight value DW_1 according to a Winograd algorithm.

[0048] Accumulator 213 accumulates the output of the processing engine circuit 212 to produce an accumulated result. In one possible embodiment, accumulator 213 writes the output of the processing engine circuit 212 into output buffer 214, and when the processing engine circuit 212 produces a new output, output buffer 214 reads the previous output, adds the new output to the previous output, and writes the sum back into output buffer 214, replacing the previous output.

[0049] Output buffer 214 stores the accumulation result of accumulator 213. In one possible embodiment, output buffer 214 writes the final accumulation result to storage circuit 120. In this example, detection circuit 132 determines the remaining amount of data, i.e., the amount of data that needs to be further convolutionally processed, based on the multiple accumulation results stored in storage circuit 120.

[0050] In some embodiments, the control circuit 131 uses enable signals SO1 to SO4 to control convolution calculation circuits CV_1 to CV_4. Taking convolution calculation circuit CV_1 as an example, when the control circuit 131 enables the enable signal SO1, convolution calculation circuit CV_1 begins to perform convolution operation on data DI_1 and weight value DW_1. When the control circuit 131 disables the enable signal SO1, convolution calculation circuit CV_1 stops performing convolution operation on data DI_1 and weight value DW_1. In other embodiments, the control circuit 131 may use the enable signal SO1 to stop providing clock signals (not shown) to input buffer 211, processing engine circuit 212, accumulator 213, and output buffer 214.

[0051] In a first processing stage, the detection circuit 132 detects an access state of the storage circuit 120 within a fixed time period to generate a detection signal SD. In one possible embodiment, the detection circuit 132 detects the idle time and idle space of the storage circuit 120 within a fixed time period. In this example, before the convolution operation of each layer, the control circuit 131 determines how many convolution calculation circuits to activate based on the idle time and idle space of the storage circuit 120 within the fixed time period. For example, when the idle time and idle space of the storage circuit 120 within the fixed time period meet a first preset condition, the control circuit 131 activates convolution calculation circuits CV_1 to CV_4. When the idle time and idle space of the storage circuit 120 within the fixed time period meet a second preset condition, the control circuit 131 activates convolution calculation circuits CV_1 to CV_3.

[0052] In the first processing stage, the amount of data to be processed is large. Therefore, the control circuit 131 activates the maximum number of convolution calculation circuits according to the access status of the storage circuit 120. The activated convolution calculation circuits perform convolution operations to generate multiple calculation results. These multiple calculation results may be written into the storage circuit 120.

[0053] In a second processing stage, the control circuit 131 or the detection circuit 132 determines the remaining amount of data from the multiple calculation results. Based on the remaining data, the control circuit 131 determines how many convolutional computation circuits to activate before the convolutional operation at each layer. In some embodiments, the control circuit 131 determines the number of convolutional computation circuits to activate based on the remaining data and the access status of the storage circuit 120. In this case, the number of activated convolutional computation circuits (or the second number) may be the same as or less than the number of activated convolutional computation circuits (or the first number) in the first processing stage.

[0054] In a third processing stage, control circuit 131 or detection circuit 132 determines the total processing time (e.g., 1 / 60 of a second) for each frame based on a frame rate. Control circuit 131 or detection circuit 132 calculates the sum of the times for the first and second processing stages. Based on the difference between the processing time (e.g., 1 / 60 of a second) and the sum of the times for the first and second processing stages, control circuit 131 determines how many convolution calculation circuits to activate before the convolution operation at each layer. In this stage, control circuit 131 may activate a third number of convolution calculation circuits. This third number is less than the first and second numbers.

[0055] In the third processing stage, in addition to considering the remaining processing time, the control circuit 131 also considers how much data (or the second operation result) remains to be processed after the convolution operation in the second processing stage. In this example, the control circuit 131 determines how many convolution calculation circuits to activate before the convolution operation of each layer in the third processing stage, based on the remaining processing time and the second operation result.

[0056] In some embodiments, the control circuit 131 includes a storage circuit 133 for storing a lookup table or a machine learning model. In this example, the control circuit 131 determines the number of convolution calculation circuits to be activated before each layer's convolution operation based on the information recorded in the storage circuit 133.

[0057] Figure 3 This invention relates to a control method for convolution operations. The control method for convolution operations of this invention can exist through program code. When the program code is loaded and executed by a machine, the machine becomes a microcontroller circuit and convolution accelerator used to implement this invention.

[0058] First, based on the output of a bus monitor, the number of convolutional computation circuits to be activated for each layer of convolutional operations in a first processing stage is determined (step S311). In one possible embodiment, the bus monitor detects an access state of a storage circuit over a fixed period of time. The bus monitor may detect the idle time and space of the storage circuit over the fixed period of time. In some embodiments, step S311 activates a first number of convolutional computation circuits to perform multi-layer convolutional operations on image data stored in the storage circuit.

[0059] Next, based on the output of the bus monitor and the remaining data volume, the number of convolutional computation circuits to be activated for each layer of convolutional operations in a second processing stage is determined (step S312). In one possible embodiment, the remaining data volume is the data volume of the operation result (or first operation result) generated by the convolutional operation in step S311. In this example, step S312 activates a second number of convolutional computation circuits to perform multi-layer convolutional operations on the operation result generated in step S311. The second number may be equal to or less than the first number.

[0060] Then, based on the frame rate and the amount of remaining data, the number of convolutional computation circuits to be activated for each layer of convolutional operations in a third processing stage is determined (step S313). In one possible embodiment, the amount of remaining data is the amount of data resulting from the convolutional operation in step S312. In this example, step S313 activates a third number of convolutional computation circuits to perform multi-layer convolutional operations on the results (or second results) generated in step S312. The third number may be less than the second number.

[0061] In some embodiments, step S313 determines a processing time based on a frame rate input by the user. Step S313 then calculates the total time of the first and second processing stages. Step S313 calculates the difference between this processing time and the total time of the first and second processing stages to determine a remaining time value. In this example, step S313 determines the number of convolutional computation circuits to be activated for each layer of convolutional computation in the third processing stage based on the amount of data in the second calculation result and the remaining time value.

[0062] Taking the Winogré algorithm as an example, the Winogré algorithm may have 16 layers of convolution operations. In this example, the first processing stage performs convolution operations from layers 1 to 4 on a set of image data. Since the amount of data for the convolution operations from layers 1 to 4 is large, step S311 activates the maximum number of convolution calculation circuits based on the idle time and free space of the storage circuit.

[0063] Next, in the second processing stage, convolution operations are performed on layers 5 through 12. At this point, the remaining data volume is determined based on the result of the convolution operation on layer 4. Based on the remaining data volume and the idle time and space of the storage circuit, an appropriate number of convolution calculation circuits are activated.

[0064] Finally, in the third processing stage, convolution operations are performed on layers 13 to 16. At this point, the remaining data volume is determined based on the result of the convolution operation on layer 12. Based on the remaining data volume and the remaining time, the minimum number of convolution calculation circuits are activated. Since the data volume for convolution operations on layers 13 to 16 is the smallest, the minimum number of convolution calculation circuits activated is sufficient to complete these operations within the remaining time. Therefore, the power consumption of the third processing stage is the lowest.

[0065] By activating appropriate convolutional computation circuits at different processing stages, overall hardware utilization can be improved. For example, when the bandwidth of the storage circuitry is sufficient, more convolutional computation circuits can be activated to achieve high efficiency. Depending on the amount of remaining data, fewer convolutional computation circuits can be activated to reduce power consumption.

[0066] Enable means changing the state of a Boolean signal. A Boolean signal can be enabled to be high or have a higher voltage, and can also be enabled to be low or have a lower voltage at the discretion of the circuit designer. Similarly, disable means changing the state of a Boolean signal to a voltage potential opposite to its enabled state.

[0067] The control method for convolution operations of the present invention, or a specific form or part thereof, can exist in the form of program code. The program code can be stored on physical media, such as floppy disks, optical disks, hard disks, or any other machine-readable (e.g., computer-readable) storage media, or not limited to external computer program products. When the program code is loaded and executed by a machine, such as a computer, this machine becomes a participant in the microcontroller circuitry and convolution accelerator of the present invention. The program code can also be transmitted via some transmission medium, such as wires or cables, optical fibers, or any transmission method. When the program code is received, loaded, and executed by a machine, such as a computer, this machine becomes a participant in the microcontroller circuitry and convolution accelerator of the present invention. When implemented in a general-purpose processing unit, the program code, combined with the processing unit, provides a unique device that operates similarly to an application-specific logic circuit.

[0068] Unless otherwise defined, all terms herein (including technical and scientific terms) are as commonly understood by those skilled in the art. Furthermore, unless expressly stated otherwise, definitions of terms in general dictionaries should be interpreted as consistent with their meaning in the context of their respective technical fields, and not as idealized or overly formal expressions. While terms such as “first” and “second” can be used to describe various elements, these elements should not be limited by these terms. These terms are merely used to distinguish one element from another. In the claims, terms such as “first” and “second” are used as designations and are not intended to impose numerical requirements on their contents.

[0069] While the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Any person skilled in the art can make modifications and refinements without departing from the spirit and scope of the invention. For example, the systems, apparatus, or methods described in the embodiments of the present invention can be implemented in physical embodiments of hardware, software, or a combination of hardware and software. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A convolution accelerator, characterized in that, Used to perform multi-layer convolution operations on data stored in a memory circuit, and includes: Multiple convolution calculation circuits, each performing a convolution operation when activated; A detection circuit detects an access state of the storage circuit within a fixed time period to generate a detection signal; and A control circuit dynamically adjusts the number of convolution calculation circuits to be activated for each layer's convolution operation based on the detection signal.

2. The convolution accelerator as described in claim 1, characterized in that, This access status refers to the idle time and free space of the storage circuit.

3. The convolution accelerator as described in claim 2, characterized in that, When the idle time and idle space of the storage circuit within the fixed time period meet a first preset condition, the control circuit activates all convolution calculation circuits. When the idle time and idle space of the storage circuit within the fixed time period meet a second preset condition, the control circuit activates some convolution calculation circuits.

4. The convolution accelerator as described in claim 1, characterized in that: In a first processing stage, based on the detection signal, the control circuit activates a first number of convolutional computation circuits among the plurality of convolutional computation circuits before the convolutional operation of each layer. These first number of convolutional computation circuits generate a first computation result. In a second processing stage, the control circuit, based on the first calculation result, activates a second number of convolution calculation circuits among the plurality of convolution calculation circuits before the convolution calculation of each layer.

5. The convolution accelerator as described in claim 1, characterized in that, Each convolution calculation circuit includes: An input buffer stores an input feature map and multiple weight values; A processing engine circuit uses a Winograd algorithm to calculate the input feature map and the multiple weight values ​​to generate multiple output results; An accumulator that accumulates the plurality of output results to produce an accumulated result; and An output buffer stores the accumulated result.

6. A microcontroller circuit, characterized in that, include: A sensing circuit detects an external image to generate input data; A storage circuit is used to store the input data; A convolution accelerator performs multiple convolution operations on the input data, and includes: Multiple convolution calculation circuits, each performing a convolution operation when activated; A detection circuit detects an access state of the storage circuit within a fixed time period to generate a detection signal; and A control circuit dynamically adjusts the number of convolution calculation circuits to be activated for each layer's convolution operation based on the detection signal; and A processing circuit is used to trigger the convolution accelerator and identify the external image based on the calculation results of the plurality of convolution calculation circuits.

7. The microcontroller circuit as described in claim 6, characterized in that, The processing circuit performs a fully connected layer operation on the computation results of the multiple convolution calculation circuits.

8. The microcontroller circuit as described in claim 6, characterized in that: In a first processing stage, based on the detection signal, the control circuit activates a first number of convolutional computation circuits among the plurality of convolutional computation circuits before the convolutional operation of each layer. These first number of convolutional computation circuits generate a first computation result. In a second processing stage, based on the amount of data from the first calculation result, the control circuit activates a second number of convolution calculation circuits among the plurality of convolution calculation circuits before the convolution operation at each layer. This second number of convolution calculation circuits generates a second calculation result. In a third processing stage, the control circuit determines a processing time based on a frame rate and calculates the total time of the first and second processing stages. Based on the difference between the processing time and the total time, the control circuit activates a third number of convolution calculation circuits among the plurality of convolution calculation circuits before the convolution operation of each layer.

9. The microcontroller circuit as described in claim 8, characterized in that: In this second processing stage, the control circuit activates a second number of convolution calculation circuits among the plurality of convolution calculation circuits based on the detection signal and the data volume of the first calculation result. In the third processing stage, the control circuit activates a third number of convolution calculation circuits among the plurality of convolution calculation circuits based on the amount of data in the second calculation result and the difference between the processing time and the total time.

10. A method for controlling convolution operations, characterized in that, include: Store image data in a storage circuit; Monitor the access status of the storage circuit within a fixed time period; Based on this access state, the number of convolutional computation circuits to be activated for each layer in a first processing stage is determined. Detect the first calculation result produced by the first processing stage; Based on the amount of data in the first calculation result and the access status, the number of convolution calculation circuits to be activated for each layer in a second processing stage is determined. Detect the result of a second operation generated in the second processing stage; Detect the frame rate of an image to determine the processing time; Calculate the total time for the first and second processing stages; Calculate the difference between this processing time and the total processing time; as well as Based on the amount of data from the second operation result and the difference, the number of convolution calculation circuits to be activated for each layer in a third processing stage is determined.

Images