Modular CNN Object Detection for Memory-Limited Microcontrollers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing microcontrollers lack sufficient on-chip memory to run high-parameter convolutional neural networks like MobilenetV2, limiting their ability to perform complex image processing tasks such as object detection, and require extensive labeling of training data, which is labor-intensive and time-consuming.
Innovation Solution
Divide the convolutional neural network into modules, pre-train a feature extraction module using unlabeled data, and then combine it with detection head modules for supervised learning with minimal labeled data, allowing the network to be run efficiently on microcontrollers with limited memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the number of convolutional layers in the CNN model is increased to improve representation ability, then the model accuracy is improved, but the number of parameters increases beyond the available on-chip memory of microcontrollers
Solution Approach 1:
The patent divides the CNN model into multiple modules (e.g., stem module, repeated modules with convolutional layers, pooling layers, and detection head modules). Each module's parameters are stored separately in on-chip memory, allowing the system to run complex models by loading only the necessary module parameters into the limited 256-512 KB on-chip memory, while larger models can utilize external memory storage.
Solution Approach 2:
The patent introduces a multi-scale detection dimension by adding detection head modules at different scales (e.g., P3, P4, P5 levels) to the CNN architecture. This allows the model to detect objects of varying sizes simultaneously without requiring a single massive model, thereby achieving high accuracy for multi-scale objects while keeping individual module parameters within memory constraints.
2Measurement precision
If traditional supervised learning is used to train the CNN model, then high precision can be achieved, but a large amount of labeled training data is required, which is labor-intensive and time-consuming
Solution Approach 1:
The patent applies preliminary unsupervised pre-training to the feature extraction modules using unlabeled data before supervised fine-tuning. This pre-training phase allows the model to learn general feature representations without requiring labeled data, reducing the amount of labeled training data needed subsequently while maintaining high detection precision.
3Measurement precision
If a complete CNN model is trained and deployed, then high detection precision is achieved, but the memory consumption exceeds the on-chip memory capacity of microcontrollers
Solution Approach 1:
The patent segments the complete CNN model into multiple independent modules (stem module, repeated modules, pooling layers, detection head modules), where each module's parameters occupy a manageable portion of on-chip memory. This segmentation allows the system to deploy high-precision models on memory-constrained microcontrollers by loading only necessary modules into 256-512 KB on-chip memory.
Solution Approach 2:
The patent designs universal repeated modules that can be instantiated multiple times with different configurations (e.g., different convolutional layer settings) to create detection head modules for different scales. These modular components can be shared across multiple detection tasks, reducing overall memory requirements while maintaining high detection precision for various object scales.
Data Source
AI summary
This invention provides a model generation method, an object detection method, a controller, and an electronic device. The model generation method comprises: constructing a convolutional neural network model used for multi-scale object detection, and dividing the convolutional neural network model into a plurality of modules, the plurality of modules comprising a feature extraction module and a plurality of detection head modules of different scales; using unlabeled training data to pre-train the feature extraction module to obtain parameters and models of the feature extraction module; and connecting the trained feature extraction module to the plurality of detection head modules respectively, and using labeled training data to train a plurality of the modules which have been connected, to obtain parameters and models of the modules. A high-precision convolutional neural network model can be obtained without the need to label a large amount of training data, and the labor and time required for labeling the training data are saved.