A deep learning model context adaptive compression method for microcontroller platform

Abstract

The present application relates to a kind of microcontroller platform-oriented deep learning model situation self-adaptive compression method, first, based on lightweight compression structure block loading and voltage frequency adjustment design, build up the adaptive joint strategy space of the structure compression and running mode of deep learning model and its intelligent application. Then, perceive microcontroller dynamic situation context-including user situation and resource situation.Finally, in the strategy search space, the solution is solved using the idea of constrained multi-objective optimization, and the structure compression and running optimization of deep learning model and its intelligent application are situation self-adaptive. The method of the present application perceives dynamic situation context, and uses lightweight compression structure block loading and voltage frequency adjustment, jointly generates the compression and running scheme of deep learning model and its intelligent application under corresponding situation, and improves the deployment and dynamic adaptive ability of model in microcontroller.

Description

Technical Field

[0001] This invention is a context-adaptive compression method for deep learning models on microcontroller platforms, involving context-adaptive structural compression and runtime optimization of deep learning models and their associated intelligent applications. Background Technology

[0002] In recent years, deep learning models have gradually shone in many artificial intelligence fields (such as speech recognition, computer vision, and natural language processing). However, their time and space complexity have also exploded, increasing the difficulty of deployment on resource-constrained mobile or embedded edge platforms. In the context of pursuing ubiquitous computing, microcontrollers (or single-chip microcomputers) have become one of the most crucial deployment platforms for future intelligent applications due to their advantages such as small size, low cost, low power consumption, and high integration. However, compared to mobile devices such as smartphones, microcontrollers, with their extremely limited storage and computing resources (for example, the mainstream STM32 series often only has <1MB of SRAM and <2MB of Flash memory), significantly increase the difficulty of model deployment. Furthermore, in addition to the challenge of resource constraints, deep learning models and their intelligent applications are constantly in dynamic user contexts (accuracy, latency, power consumption requirements, etc.) and resource contexts (power consumption, storage space, etc.). Traditional models, however, are often trained based on specific datasets and fixed topologies, lacking dynamic mechanisms and exhibiting poor adaptability to different contexts.

[0003] In existing technologies, a series of resource-friendly model studies (such as model compression and lightweight network design), neural network architecture search techniques, and dynamic neural network inference have made it possible to successfully and flexibly deploy deep learning models on resource-constrained platforms. For example, Han Song's team at MIT proposed MCUNet, which co-designs neural network architecture search and inference engines for microcontroller platforms, further improving the efficiency of deep learning model implementation on micro devices. BlockDrop, through a separate policy network, adaptively selects the inference path of the backbone network based on the complexity of the input samples during inference. However, these technologies either require time-consuming retraining or architecture search steps, lacking an adaptive foundation for fast, low-cost online tuning; or they lack context-adaptive capabilities, failing to consider the dynamic changes of the model and the intelligent application context. Summary of the Invention

[0004] Technical problems to be solved

[0005] To overcome the shortcomings of existing technologies, this invention provides a context-adaptive compression method for deep learning models on microcontroller platforms. Designed for microcontrollers (or single-chip microcomputers)—a crucial future deployment platform for intelligent applications—this method addresses the challenges of deploying deep learning models due to extreme resource constraints and the poor adaptability of traditional deep learning models to the dynamic characteristics of deployment scenarios.

[0006] Technical solution

[0007] A context-adaptive compression method for deep learning models on microcontroller platforms, characterized by the following steps:

[0008] Step 1: Design a lightweight compressed block loading design and a voltage-frequency configuration pair for the microcontroller platform;

[0009] Step 2: Sensing the dynamic situation of the microcontroller, including the current remaining RAM(t), remaining persistent storage Flash(t), and latency requirement L. threshold (t) and platform power;

[0010] Step 3: Model the scenario-adaptive soft-hard co-policy search process for microcontrollers:

[0011] m = argmin aεA，s∈S L(B, a, s)

[0012] L(B,a,s)=λ1(t)Norm((1-Acc(B,a)))+λ2(t)Norm(EE(B,a,s))

[0013] stPMU(B,a)<RAM(t), MS(B,A)<Flash(t), Latency(B,a,s)<L threshold (t)

[0014] The above equation models the optimization problem of adapting strategy search under specific contextual indicators by setting model accuracy and application energy consumption as optimization objectives, and RAM, Flash, and latency requirements as constraints. Here, A is the compression operator group space, a is the current compression operator selection scheme; S is the voltage frequency adjustment space, s is the current voltage frequency configuration scheme, and m represents the currently selected (a, s) joint strategy; L represents the constrained multi-objective function for determining the joint strategy, B represents the selected backbone network, and L... threshold (t) represents the current time delay budget threshold; <λ1(t), λ2(t)> are the importance coefficients that change over time, reflecting the trade-offs between model accuracy and running energy consumption at different times. They depend on user needs or changes in platform resources. Norm(·) represents the standardization operation of non-uniform indicators of the objective function.

[0015] Step 4: After completing the deployment of the variant model group and voltage-frequency configuration pairs consisting of lightweight compressed structural blocks and modeling the adaptive strategy search process, the joint adaptation strategy is based on the dynamic situation perceived in Step 2.

[0016] A further technical solution of the present invention: Step 1 is as follows:

[0017] First, based on the analysis of the actual overhead of the model and the inherent resources of the microcontroller, a backbone network model is designed. Then, at the model level, a lightweight compressed structure block loading design is used, and different types of lightweight convolutional computation modes are adopted to replace traditional convolutional computation, thus constructing a variant model group space A with different performance characteristics. 8-bit quantization technology is then used to deploy the backbone network and lightweight compressed structure blocks on the microcontroller. Simultaneously, at the platform level, discrete voltage-frequency pairs are designed according to the specific operating rules of the microcontroller, enabling intelligent applications on the platform to exhibit different latency and energy consumption characteristics, thus constructing a voltage-frequency configuration space S. Finally, the two levels are combined to form a hardware-software co-operated adaptive policy space M = (A, S), which serves as the basis for subsequent contextual policy search.

[0018] A further technical solution of the present invention: The lightweight convolutional computation mode includes depthwise separable convolutional blocks, Fire structure blocks in SqueezeNet, low-rank decomposition structure blocks based on singular value decomposition, and multi-branch parallel structure blocks.

[0019] A computer system is characterized by comprising: one or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method described above.

[0020] A computer-readable storage medium is characterized by storing computer-executable instructions, which, when executed, are used to implement the above-described method.

[0021] Beneficial effects

[0022] This invention provides a context-adaptive compression method for deep learning models on microcontroller platforms. First, based on lightweight compression block loading and voltage / frequency adjustment design, an adaptive joint policy space for the structural compression and operational modes of the deep learning model and its associated intelligent application is constructed. Each policy has different practical performance characteristics (such as accuracy, storage footprint, latency, and energy consumption). Next, the dynamic context of the microcontroller—including user and resource contexts—is perceived. Finally, within the policy search space, a constrained multi-objective optimization approach is used to solve for a solution, performing context-adaptive structural compression and operational tuning of the deep learning model and its associated intelligent application. This invention perceives dynamic context and utilizes lightweight compression block loading and voltage / frequency adjustment to jointly generate compression and operational schemes for the deep learning model and its associated intelligent application under corresponding contexts, enhancing the model's deployment and dynamic adaptive capabilities on the microcontroller. Attached Figure Description

[0023] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0024] Figure 1 This is a schematic framework of the context-adaptive compression method for deep learning models on microcontroller platforms according to the present invention.

[0025] Figure 2 This is a schematic diagram illustrating the construction of a variant model group space based on the lightweight compressed structural block loading design of the present invention and its deployment on a microcontroller. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0027] This invention provides a context-adaptive compression method for deep learning models on microcontroller platforms, comprising:

[0028] Software-hardware co-adaptive policy space: The joint adaptive policy space of the model and its intelligent application is built by using lightweight compressed structural block loading and discrete voltage and frequency configuration.

[0029] Dynamic context awareness: It senses the user context (accuracy, latency, power consumption requirements, etc.) and resource context (platform power consumption, SRAM running memory, Flash persistent storage, etc.) of the deployment of deep learning models on the microcontroller platform.

[0030] Context-adaptive search: Within the policy space, deployment schemes are solved using constrained multi-objective optimization, and the deep learning model and its associated intelligent application are structurally compressed and optimized based on the context.

[0031] like Figure 1 and Figure 2 As shown, the specific steps of the present invention are as follows:

[0032] Step 1: For the microcontroller platform, design lightweight compressed block loading and voltage-frequency configuration pairs to complete the construction and deployment of the joint adaptive policy space for the model and its intelligent application. Variant models formed by different loading schemes avoid accuracy interference through ensemble training. Discrete voltage-frequency configuration pairs must meet the operating rules of the microcontroller platform (generally, voltage can only be adjusted to the next stage when the frequency drops to a certain threshold). Specifically, firstly, based on the analysis of the model's actual overhead and the microcontroller's inherent resources, a backbone network model is designed. Then, at the model level, using the lightweight compressed block loading design, different types of lightweight convolutional computation modes (such as depthwise separable convolutional blocks, Fire blocks in SqueezeNet, low-rank decomposition blocks based on singular value decomposition, and multi-branch parallel blocks, etc.) are used to replace traditional convolutional computation, realizing the construction of a variant model group space A with different performance characteristics (accuracy, latency, energy consumption, memory usage, persistent storage usage, etc.). 8-bit quantization technology is used to complete the deployment of the backbone network and lightweight compressed blocks on the microcontroller (accuracy interference between different loading schemes is avoided through ensemble training). Simultaneously, at the platform level, discrete voltage-frequency pairs are designed according to the specific operating rules of the microcontroller. This enables intelligent applications on the platform to exhibit different latency and energy consumption characteristics, constructing a voltage-frequency configuration space S. Finally, the two levels are combined to form a hardware-software co-operational adaptive policy space M = (A, S), serving as the basis for subsequent contextual policy search. The design and policy selection of the two spaces are relatively independent and do not affect each other, exhibiting strong scalability.

[0033] Step 2: Sensing the dynamic situation of the microcontroller, including the current remaining RAM(t), remaining persistent storage Flash(t), and latency requirement L. threshold (t) and platform power.

[0034] Step 3: Model the scenario-adaptive soft-hard co-policy search process for microcontrollers:

[0035] m = argmin a∈A，s∈S L(B, a, s)

[0036] L(B,a,s)=λ1(t)Norm((1-Acc(B,a)))+λ2(t)Norm(EE(B,a,s))

[0037] stPMU(B,a)<RAM(t), MS(B,A)<Flash(t), Latency(B,a,s)<L threshold (t)

[0038] The above equation models the optimization problem of adapting strategy search under specific contextual indicators by setting model accuracy and application energy consumption as optimization objectives, and RAM, Flash, and latency requirements as constraints. Here, A is the compression operator group space, a is the current compression operator selection scheme; S is the voltage frequency adjustment space, s is the current voltage frequency configuration scheme, and m represents the currently selected (a, s) joint strategy. In the equation, L represents the constrained multi-objective function that determines the joint strategy, B represents the selected backbone network, and L... threshol λ(t) represents the current time-delay budget threshold. <λ1(t), λ2(t)> are time-varying importance coefficients, reflecting the trade-offs between model accuracy and operating energy consumption at different times, depending on user needs or platform resource changes (such as power thresholds). Furthermore, Norm(·) represents the standardization operation (e.g., logarithmic operation) of non-uniform metrics of the objective function.

[0039] Step 4: After completing the deployment of the variant model group and voltage-frequency configuration pairs consisting of lightweight compressed structural blocks and modeling the adaptive strategy search process, generate a joint adaptation strategy based on the dynamic context and the intelligent application in which it is located (such as the operation and application of the generated model, such as food detection, sound detection, and garbage recognition).

[0040] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the scope of the technology disclosed in the present invention, and such modifications or substitutions should all be covered within the scope of protection of the present invention.

Claims

1. A context-adaptive compression method for deep learning models on microcontroller platforms, characterized in that... The steps are as follows: Step 1: Design a lightweight compressed block loading design and a voltage-frequency configuration pair for the microcontroller platform; Step 2: Sensing the dynamic situation of the microcontroller, including the current remaining running memory. RAM(t) Remaining persistent storage Flash (t) Latency requirements and platform battery level; Step 3: Model the scenario-adaptive soft-hard co-policy search process for microcontrollers: The above formula models the optimization problem of adaptability strategy search under specific contextual indicators by setting model accuracy and application energy consumption as optimization objectives, and RAM, Flash, and latency requirements as constraints; where, It is a compression operator group space. Select a scheme for the current compression operator; It is the voltage and frequency adjustment space. For the current voltage and frequency configuration scheme, Indicates the currently selected Joint strategy; where, Denotes a constrained multi-objective function for determining a joint strategy. Represents the selected backbone network. Indicates latency requirements; The importance coefficient, which varies over time, reflects the trade-offs between model accuracy and operational energy consumption at different times. It depends on changes in user needs or platform resources. Standardization operations representing indices of different magnitudes in the objective function; Step 4: After completing the deployment of the variant model group and voltage-frequency configuration pairs consisting of lightweight compressed structural blocks and modeling the adaptive strategy search process, the joint adaptation strategy is based on the dynamic situation perceived in Step 2.

2. The context-adaptive compression method for deep learning models on microcontroller platforms according to claim 1, characterized in that... Step 1 is as follows: First, based on the analysis of the actual overhead of the model and the inherent resources of the microcontroller, a backbone network model is designed. Then, at the model level, a lightweight compressed structural block loading design is used, and different types of lightweight convolutional computation modes are adopted to replace traditional convolutional computation, thereby realizing a variant model group space with different performance characteristics. The backbone network and lightweight compressed structural blocks were constructed and deployed on the microcontroller using 8-bit quantization technology. Simultaneously, at the platform level, discrete voltage-frequency pairs were designed according to the specific operating rules of the microcontroller, enabling intelligent applications on the platform to exhibit different latency and energy consumption characteristics, thus establishing a voltage-frequency configuration space. Ultimately, the two levels will be combined to form an adaptive strategy space that integrates software and hardware. This serves as the foundation for subsequent contextual strategy search.

3. The context-adaptive compression method for deep learning models on microcontroller platforms according to claim 2, characterized in that... The lightweight convolution computation mode includes depthwise separable convolutional blocks, Fire structure blocks in SqueezeNet, low-rank decomposition structure blocks based on singular value decomposition, and multi-branch parallel structure blocks.

4. A computer system, characterized in that... include: One or more processors, a computer-readable storage medium for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of claim 1.

5. A computer-readable storage medium, characterized in that... The device stores computer-executable instructions, which, when executed, are used to implement the method of claim 1.

Images