An intelligent monitoring tinyML method based on 4-bit quantization

Abstract

The application relates to the technical field of artificial intelligence personnel detection, in particular to an intelligent monitoring tinyML method based on 4bit quantization, which comprises the following steps: (1) training of a personnel detection model; (2) post-training quantization of the model; (3) establishment of an AI inference framework on an MCU; (4) on-chip storage of a 4bit model and on-chip storage coding of inference first weight; and (5) data acquisition of a camera and real-time inference of data. The method not only significantly improves the storage and operation efficiency in a tinyML device, but also optimizes the inference framework, which makes it possible to realize a complex machine learning model in a resource-limited environment. The optimizations ensure the competitiveness of the application in tinyML applications and provide a practical and cost-effective solution.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence personnel detection, and particularly relates to an intelligent monitoring tinyML method based on 4bit quantization. BACKGROUND

[0002] In the field of artificial intelligence, especially in computer vision, personnel detection technology is a core technology, widely used in security monitoring, smart retail, human-computer interaction and other fields. With the development of deep learning technology, personnel detection models based on convolutional neural networks (CNN) have made significant improvements in accuracy. However, these models often require high computing power and storage resources, which limits their application in resource-constrained environments, such as embedded devices or mobile devices. Also includes:

[0003] 1. Computing resource limitation: existing technologies are severely limited in computing resources when processing image data, especially on embedded systems or tinyML devices;

[0004] 2. Model accuracy and inference efficiency: improving the recognition accuracy of models in real-world environments is a persistent problem.

[0005] 3. Adaptability and deployment difficulty: existing personnel detection systems are often complex to deploy;

[0006] 4. Cost and power consumption: traditional deep learning models often require a large amount of storage space, which is a challenge for devices with limited storage resources.

[0007] In the field of tinyML (Tiny Machine Learning), some famous projects and research teams have proposed and implemented personnel detection solutions for resource-constrained devices. For example:

[0008] 1. Google's TensorFlow Lite for Microcontrollers is a machine learning library designed specifically for microcontrollers and other memory-constrained devices. The project provides an end-to-end toolchain that supports starting from training large models, and then using techniques such as quantization and pruning to compress these models into versions suitable for running on microcontrollers. Google's project uses lightweight network structures such as MobileNetV2, and uses quantization techniques to compress the model to be small enough to be deployed on low-power devices such as Arduino and ESP32.

[0009] 2. ARM launched CMSIS-NN, a neural network library optimized for ARM Cortex-M series processors. CMSIS-NN improves the performance of neural network models running on ARM processors by providing low-level functions optimized for the ARM processor architecture. In relation to tinyML, this project makes it possible to implement person detection on microcontrollers like the STM32.

[0010] 3. Edge Impulse is a company focused on edge AI, providing a development platform to support developers and businesses in building, training, and deploying machine learning models on edge devices. The platform supports various tinyML applications, including person detection, allowing developers to use models like MobileNet and convert them into suitable formats for running on extremely resource-constrained hardware through techniques like quantization.

[0011] 4. OpenMV is an open-source project aimed at simplifying the development of machine vision tasks on microcontrollers. It provides an integrated development environment and a corresponding camera module that can directly run tinyML models, such as for person detection. It also integrates lightweight models like MobileNetV2 and optimizes them for microcontrollers.

[0012] In summary, while existing technologies have made some progress in the field of tinyML, these solutions prove that effective person detection on resource-constrained devices is feasible, but there are still challenges such as real-time performance, energy efficiency, and adaptability. There are some limitations and drawbacks. The main problems can be summarized as follows:

[0013] 1. Resource consumption: Although existing technologies use lightweight models and compress them through techniques like quantization, none of them successfully support 4-bit weight models. The lowest supported inference is 8-bit or 16-bit. In some extremely resource-constrained scenarios, the resource consumption of these models is still relatively high, which may not meet the needs of extremely low power consumption or ultra-small flash.

[0014] 2. Inference efficiency: In existing technical solutions, even compressed models may have delays during real-time inference, especially when processing complex or high-resolution input data. This limits their use in applications that require fast responses.

[0015] 3. Adaptability and scalability: TinyML solutions on the market often require developers to have some knowledge of hardware to effectively deploy them, which limits the scope of application and ease of use of these technologies.

[0016] 4. Cost issue: Although hardware costs are decreasing, some solutions may require higher-performance chips to meet real-time inference requirements, which increases the cost of the entire system.

[0017] Therefore, a new technical solution is urgently needed to solve the above technical problems. Summary of the Invention

[0018] The purpose of the present invention is to overcome the problems of the above-mentioned prior art and provide an intelligent monitoring tinyML method based on 4-bit quantization to solve the technical problems of large resource consumption, low inference efficiency, poor adaptability and scalability, and high cost in the existing technology.

[0019] The above objectives are achieved through the following technical solutions:

[0020] A 4-bit quantization-based intelligent monitoring TinyML method, including:

[0021] Step (1) Training of personnel detection model

[0022] First, prepare and preprocess the data set, select and customize the person detection model, and then train and verify the person detection model;

[0023] The selection and customization of the personnel detection model adopts the MobileNetV2 structure to reduce the complexity by using depthwise separable convolution. The formula is as follows:

[0024] Y = DConv(X)·PConv(DConv(X))

[0025] Among them, DConv is depth convolution, PConv is point convolution, X and Y represent input and output features respectively;

[0026] The output layer of the MobileNetV2 structure was also adjusted, and the final fully connected layer formula was modified to:

[0027] Y custom =FC custom (X)

[0028] Among them, FC custom It is a tuned fully connected layer, specially customized for person detection.

[0029] The training and verification of the personnel detection model uses binary cross entropy as the loss function, and the formula is as follows:

[0030]

[0031] in, is the model for x ithe predicted value of the model;

[0032] The optimizer selects Adam, whose parameter update rule is as follows:

[0033]

[0034] where θ represents the model parameters, η is the learning rate, and are the first and second moment estimates of the gradient, respectively, and ∈ is a small constant to prevent division by zero operations;

[0035] Step (2) post-training quantization of the model

[0036] First, 4-bit linear quantization is performed, and then quantization calibration of the activation function is performed, bias_correction bias correction;

[0037] The linear transformation formula of the linear quantization is as follows:

[0038]

[0039] where x represents the original high-precision parameter, Q(x) represents the quantized low-precision parameter, Δ represents the quantization interval, and Z is the quantization zero-point offset; the quantization operation uses rounding to minimize quantization error;

[0040] In the context of 4-bit quantization, the determination of the quantization interval Δ and the quantization zero point Z is crucial; the quantization interval is calculated by the following formula:

[0041]

[0042] For 4-bit quantization, n = 4, so the denominator is 2 4 -1 = 15; where max(x) and min(x) represent the maximum and minimum values, respectively, in the data range under consideration for quantization;

[0043] After determining the quantization interval, the zero-point offset Z is set as:

[0044]

[0045] The quantization function can be further specified as:

[0046]

[0047] Step (3) building of the AI inference framework on the MCU

[0048] First, increase the throughput of the system, then reduce the algorithm complexity, and then adjust the performance of the inference engine;

[0049] Step (4) on-chip storage of 4bit model and inference

[0050] First, the on-chip storage encoding of the weight is performed, and then the inference framework is adjusted; Specifically, the model weight which can only be stored in the form of 8bit at the lowest level is encoded by encoding two adjacent 4bit weights into an 8bit unsignedchar type. After adjusting the inference framework, the weight decoding will decode a new array w_ram[] as the weight of this layer, which is input into the convolution operator together with the output of the activation function of the previous layer for convolution operation, and the output obtained after the activation function is used as the activation input of the next layer.

[0051] In the inference process, since the multiplier in the MCU accepts the input of 8bit at the lowest level, a decoding module is designed, which decodes layer by layer. The inference framework calls the decoding module to convert the model weight from 4bit compression format to 8bit format required for inference.

[0052] In the inference function, before calling the convolution operator of each layer, the model weight will be prepared by the decoding module before each layer inference, and after the dequantization, it is ensured that the weight is input to the convolution operator in the correct format in the inference process, and it can be correctly accessed and processed.

[0053] Step (5) data acquisition of the camera and real-time inference of the data

[0054] First, the spi transmission of the camera data is performed, then the reshape processing of the collected data is performed, and finally the real-time inference of the data is performed.

[0055] Further, the training of the personnel detection model in step (1) further includes introducing transfer learning in the training process, and the pre-trained weight W pre is used to realize it, and the formula is as follows:

[0056] W 1:k =W pre,1:k

[0057] And W k+1:N is updated in the training process.

[0058] Finally, in order to prevent overfitting, a regularization term R(W) is introduced into the loss function, and the formula is as follows:

[0059] L reg =L+λR(W)

[0060] Here λ is the regularization coefficient, and R(W) is the regularization term, which is used to punish the size of the model weight to improve the generalization ability of the model.

[0061] Further, the linear quantization in step (2) further comprises a dequantization operation, which is formulated as follows:

[0062] x dequant = (Q(x) - Z) · Δ + min(x)

[0063] Dequantization allows the quantized values to be converted back to the approximate original floating-point values for subsequent calculations.

[0064] Further, the quantization calibration of the activation function in step (2) comprises activation distribution collection, quantization range determination, and quantization parameter determination, wherein:

[0065] The activation distribution collection specifically comprises, before quantization, first running the forward propagation of the model to collect the activation function output values of different layers; collecting the statistical distribution of these output values, including their minimum value, maximum value, mean value, and standard deviation; these statistics will be used for subsequent quantization parameter determination;

[0066] The quantization range determination specifically comprises determining a quantization range according to the statistical distribution of the activation function output values;

[0067] The quantization parameter determination specifically comprises calculating the quantization step size Δ and the zero point Z using the determined quantization range, which is formulated as follows:

[0068]

[0069]

[0070] wherein Range is the quantization range determined based on the statistical data, and n is the quantization bit width.

[0071] Further, in the quantization process, especially in low-bit-width quantization strategies such as 4-bit quantization, the quantization of weights and activations often leads to systematic bias of the model; the bias correction mechanism comprises a mean shift correction coefficient μ c and a quantization scale factor ξ c ; the mean shift correction coefficient measures the difference between the actual mean value of the weight W c and the mean value of the quantized weight ; its calculation expression is as follows:

[0072]

[0073] wherein represents the expected value or average value of the weight W c , and is the expected value of the quantized weight ;

[0074] The parameter of quantization scale factor is measured by comparing the norm distribution of weights before and after quantization, to measure the impact of quantization operation on weight scale; the formula is defined as follows:

[0075]

[0076] Here, W c denotes the original weight W c , and opt denotes the L2 norm of W base relative to its mean. denotes the quantized weight

[0077] Further, the formula for increasing the throughput of the system in step (3) is as follows:

[0078]

[0079] where T opt denotes the optimized system throughput, T base is the baseline throughput, N represents the number of processor cores, and L opt is the optimized processing delay; by parallelizing operations and reducing single-core processing delay, the overall throughput is increased.

[0080] Further, the formula for reducing algorithm complexity in step (3) is as follows:

[0081] C red = C orig ×(1-R)

[0082] where C red is the optimized algorithm complexity, C orig is the original complexity, and R is the complexity reduction ratio; an efficient quantization algorithm is used to reduce computational requirements.

[0083] Further, the formula for adjusting the performance of the inference engine in step (3) is as follows:

[0084] P adp = f(C,F)

[0085] where P adp represents the adapted inference performance, C is the cache size, and F is the processor frequency; the inference engine settings are adjusted according to hardware characteristics to adapt to different workloads.

[0086] Further, the step (4) also provides a Python script for storing the generated 4-bit weights in the form of the above-mentioned encoding in the const unsigned char array. This script simplifies the process of converting the quantized 4-bit weights into the 4-bit compression format used in the present application.

[0087] Further, the reshape processing of the collected data in step (5) includes the following steps:

[0088] (5-1) Data buffering: The original image data collected by the camera is first stored in a temporary buffer;

[0089] B(x, y) = I(x, y)

[0090] As shown in the above formula, the data buffering step, where B(x, y) represents the pixel value in the buffer, I(x, y) represents the pixel value at the corresponding position in the original image, is to copy the original image data to the buffer;

[0091] (5-2) Data preprocessing:

[0092]

[0093] The above formula is for image data normalization, which maps the pixel value I(x, y) to a unified range, usually [0.1]; here max(I) and min(I) represent the maximum and minimum values in the image data, respectively;

[0094] (5-3) Dimension conversion: For image data of a monitoring system, dimension conversion involves converting two-dimensional image data into a three-dimensional tensor compatible with the model input; this step is achieved through a dimension transformation algorithm that rearranges the original two-dimensional image matrix into the required height, width, and channel number for the model; the process of dimension conversion is as follows:

[0095] T h wc = reshape(I H w , (H, W, C));

[0096] Without changing the pixel values, only changing their arrangement in memory; where I H w is the original two-dimensional image data, T h wc is the converted three-dimensional tensor, H, W, and C represent the height, width, and channel number of the image, respectively.

[0097] The intelligent monitoring tinyML method based on 4bit quantization provided by the present application not only significantly improves the storage and operation efficiency in the tinyML device, but also optimizes the inference framework, making it possible to implement complex machine learning models in resource-constrained environments. These optimizations ensure the competitiveness of the present application in tinyML applications, providing a practical and cost-effective solution. In addition, the following advantages exist:

[0098] 1. The present patent technology ensures that efficient 4bit quantization model inference can be effectively performed on ARM architecture with limited resources, suitable for real-time data processing and machine learning;

[0099] 2. By accurately calculating μ c and ξ c , the present patent technology ensures that quantization operations will not significantly negatively affect the performance of the model. This method is particularly important for tinyML systems, which often run in resource-constrained environments, and every bit of performance optimization can have a significant effect;

[0100] 3. The bias correction technology in the present patent effectively reduces the impact of quantization errors on the final performance by accurately adjusting the bias parameters of the quantization network. The innovation of this technology lies in its ability to not only correct the bias at the model level, but also optimize it for hardware implementation characteristics to achieve efficient and accurate operation in resource-constrained tinyML systems. Through this method, the present patent ensures that even under strict 4bit quantization constraints, the system can still achieve similar inference accuracy to the original model, thereby promoting the practical application of tinyML technology in intelligent monitoring;

[0101] 4. Using KL divergence as the optimization target for activation function quantization calibration, the present patent can finely adjust the quantization parameters to ensure that even under strict 4bit quantization constraints, the inference performance can be as close as possible to that of the original floating-point model. This method improves hardware efficiency while maximizing model accuracy, providing strong technical support for real-time applications of tinyML systems. BRIEF DESCRIPTION OF DRAWINGS

[0102] Figure 1 A flowchart of the intelligent monitoring tinyML method based on 4bit quantization described in the present application;

[0103] Figure 2 A schematic diagram of on-chip storage and inference process of the 4bit model in the intelligent monitoring tinyML method based on 4bit quantization described in the present application. DETAILED DESCRIPTION

[0104] The application will be described in further detail below with reference to the drawings and embodiments. The described embodiments are only a part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the application.

[0105] As shown in the scheme, Figure 1 the present scheme provides a 4bit quantization-based intelligent monitoring tinyML method, comprising:

[0106] Step (1) training of a personnel detection model

[0107] First, data set preparation and preprocessing, as well as personnel detection model selection and customization, are performed, and the personnel detection model is trained and verified;

[0108] In the data set preparation and preprocessing stage, the data set uses a VWW data set (Visual Word2Vec dataset), which is a data set for image classification and retrieval, containing a large number of pictures, labeled as whether a human appears; the data set division formula is:

[0109]

[0110] where x i is the i-th image, y i is the corresponding label, when y i = 1, it means that there is a person in the image, and when y i = 0, it is the opposite;

[0111] Then data cleaning, which aims to remove noise and irrelevant information in the image, usually needs to correct the wrong label, the formula is as follows:

[0112] y′ i = f(x i )

[0113] where f(x i ) is the correction function;

[0114] Data augmentation is a key step to reduce overfitting and improve model generalization, which is achieved by applying a series of transformations, the formula is as follows:

[0115] x′ i = g(x i )

[0116] Here g(x i ) represents the operation of data augmentation, which can include rotation, scaling, and flipping, etc.

[0117] The selection and customization of the personnel detection model adopts the MobileNetV2 structure, which reduces complexity by using depth separable convolution, as follows:

[0118] Y = DConv(X) PConv(DConv(X))

[0119] Where DConv is the depth convolution, PConv is the point convolution, and X and Y represent the input and output features, respectively.

[0120] To adapt to the binary classification task, the output layer of the MobileNetV2 structure is also adjusted, and the final fully connected layer formula is modified as follows:

[0121] Y custom = FC custom (X)

[0122] Where FC custom is the adjusted fully connected layer, specially customized for personnel detection.

[0123] The training and verification of the personnel detection model use binary cross-entropy as the loss function, as follows:

[0124]

[0125] Where, is the predicted value of the model for x i ;

[0126] The optimizer selects Adam, and the parameter update rule formula is as follows:

[0127]

[0128] Where θ represents the model parameters, η is the learning rate, and are the first and second moment estimates of the gradient, respectively, and ∈ is a small constant to prevent division by zero operations;

[0129] During the training process, transfer learning is introduced, which is implemented by using pre-trained weights W pre and freezing the weights of the first k layers, as follows:

[0130] W 1:k = W pre,1:k

[0131] While W k+1:N is updated during the training process.

[0132] Finally, to prevent overfitting, a regularization term R(W) is introduced to the loss function, as follows:

[0133] L reg = L + λR(W)

[0134] Here λ is the regularization coefficient, and R(W) is the regularization term, which is used to penalize the size of the model weights to improve the generalization ability of the model.

[0135] Step (2) Post-training quantization of the model

[0136] First, 4-bit linear quantization is performed, and then quantization calibration of the activation function is performed, bias_correction bias correction;

[0137] Linear quantization algorithm is the core of quantization, which realizes the mapping of high-precision model parameters to low-precision representation range. This patent adopts the following linear transformation formula to ensure that the dynamic range of the converted data is as consistent as possible with the original data: as follows:

[0138]

[0139] Where x represents the original high-precision parameter, Q(x) represents the quantized low-precision parameter, Δ represents the quantization interval, and Z is the quantization zero offset; the quantization operation uses rounding to minimize quantization error;

[0140] In the context of 4-bit quantization, the determination of the quantization interval Δ and the quantization zero Z is crucial; the quantization interval is usually calculated by the following formula:

[0141]

[0142] For 4-bit quantization, n = 4, so the denominator is 2 4 -1 = 15; where max(x) and min(x) represent the maximum and minimum values in the data range under consideration for quantization;

[0143] After determining the quantization interval, the zero point offset Z is usually set as:

[0144]

[0145] The quantization function can be further specified as:

[0146]

[0147] In this case, the value range of Q(x) is limited to 0 to 16-1, i.e. 0 to 15; this requires the original parameter x to be scaled and truncated to adapt to this limited representation range;

[0148] After quantization, in order to use the quantized values during inference, a dequantization operation is usually required, the formula is as follows:

[0149] x dequant = (Q(x) - Z) · Δ + min(x)

[0150] Dequantization allows the quantized values to be converted back to approximate original floating-point values for subsequent calculations.

[0151] Given that 4-bit quantization greatly limits the representation range, special attention is paid to avoid the impact of outliers on the entire quantization range; therefore, clipping or outlier handling techniques can be adopted to determine a reasonable clipping threshold T, and then limit the x value within the range of [-T, T].

[0152] In addition, in order to further optimize the quantization process, the present patent proposes an adaptive quantization strategy that automatically adjusts the quantization parameters based on fine-grained analysis of data distribution to ensure uniform and effective representation in each data distribution segment. Please note that the specific implementation of quantization may vary depending on different model architectures, data sets and performance indicators. The method of the present patent will be optimized for each of the above to ensure that the quantized model can significantly reduce model size and improve operation efficiency while maintaining original accuracy.

[0153] The quantization calibration of the activation function includes activation distribution collection, quantization range determination and quantization parameter determination, wherein:

[0154] The activation distribution collection specifically refers to first running the forward propagation of the model before quantization, collecting the activation function output values of different layers; collecting the statistical distribution of these output values, including their minimum value, maximum value, mean value and standard deviation; these statistics will be used for subsequent quantization parameter determination;

[0155] The determination of the quantization range specifically refers to determining a suitable quantization range according to the statistical distribution of the activation function output values; this usually involves selecting a truncation threshold, for example by considering a certain percentile of the distribution of output values, such as the 99th percentile, to avoid unnecessarily expanding the quantization range due to extreme outliers.

[0156] The determination of the quantization parameter specifically refers to calculating the quantization step size Δ and the zero point Z using the determined quantization range, as follows:

[0157]

[0158]

[0159] Where Range is the quantization range determined based on statistical data, and n is the quantization bit width, which is 4 for the present solution;

[0160] Further included are optimization and calibration: based on KL divergence, the calibration process of activation function quantization adopts an optimization method based on KL divergence, which can quantitatively evaluate the difference between the distributions of activation function output values before and after quantization, and minimize this difference by optimizing the quantization parameters;

[0161] First, calculate the KL divergence between the distribution of the original activation value and the distribution of the quantized activation value; the KL divergence is defined as:

[0162]

[0163] Where P(x) is the probability distribution of the original activation value, and Q(x) is the probability distribution after quantization.

[0164] In order to calculate the KL divergence, the probability density functions of the two distributions need to be obtained or estimated. This is usually achieved by collecting a large amount of activation data and constructing a histogram. The histograms of original and quantized activation values need to use the same number of bins and the same range to ensure comparability.

[0165] By iteratively adjusting the quantization parameters (such as quantization intervals and quantization centers) and calculating the corresponding KL divergence values, the quantization parameters that minimize the KL divergence are found. This process involves building an optimization framework, such as gradient descent or other search algorithms, to effectively search the parameter space.

[0166] Once the optimal quantization parameters are found, that is, the parameters that minimize the KL divergence, the next step is to verify the performance of the quantization model on one or more independent datasets. This step is an important step to ensure that the selected parameters have generalization ability in actual application.

[0167] Finally, according to the results of KL divergence, further compensation mechanisms may be needed to correct the errors introduced by quantization. This patent proposes an advanced loss compensation algorithm to reduce the overall performance loss by fine-tuning network parameters or adaptively adjusting the quantization parameters of subsequent layers.

[0168] Using KL divergence as the optimization target of activation function quantization calibration, this patent can finely adjust the quantization parameters to ensure that even under strict 4bit quantization constraints, the inference performance can be as close as possible to the original floating-point model. This method improves hardware efficiency while maximizing model accuracy, providing strong technical support for real-time applications of tinyML systems.

[0169] In the quantization process, especially in low-bit-width quantization strategies such as 4bit quantization, the quantization of weights and activation values often leads to systematic bias in the model; the bias correction mechanism includes a mean shift correction coefficient μ c and a quantization scale factor ξc ; the mean shift correction coefficient measures the difference between the actual mean of the weight W c and the mean of the quantized weight W ; its calculation expression is as follows:

[0170]

[0171] wherein represents the expected value or mean of the weight W c , and is the expected value of the quantized weight W ;

[0172] The quantization scale factor parameter is measured by comparing the norm distribution of the weight before and after quantization, to measure the impact of the quantization operation on the weight scale; the formula definition is as follows:

[0173]

[0174] Here, represents the L2 norm of the original weight W c relative to its mean, and the L2 norm of the quantized weight W relative to its mean; the value of ξ c is used to adjust the quantized weight, so that the scale change in the quantization process can be compensated.

[0175] By accurately calculating μ c and ξ c , the present patent technology ensures that the quantization operation will not have a significant negative impact on the performance of the model. This method is particularly important for applications in tinyML systems, as these systems often run in resource-constrained environments, and every bit of performance optimization can have a significant effect.

[0176] The bias correction technology in this patent effectively reduces the impact of quantization errors on the final performance by accurately adjusting the bias parameters of the quantized network. The innovation of this technology lies in that it not only corrects the bias at the model level, but also optimizes for the characteristics of hardware implementation to achieve efficient and accurate operation in resource-constrained tinyML systems. Through this method, the present patent ensures that even under strict 4bit quantization constraints, the system can still achieve similar inference accuracy as the original model, thereby promoting the practical application of tinyML technology in the field of intelligent monitoring.

[0177] Step (3) Building of AI inference framework on MCU

[0178] First, increase the throughput of the system, then reduce the complexity of the algorithm, and then adjust the performance of the inference engine;

[0179] The formula for increasing the throughput of the system in this step is as follows:

[0180]

[0181] Where T opt represents the optimized system throughput, T base is the baseline throughput, N represents the number of processor cores, and L opt is the optimized processing delay; by parallelizing operations and reducing single-core processing delay, the overall throughput is increased.

[0182] The formula for reducing algorithm complexity in this step is as follows:

[0183] C red = C orig × (1-R)

[0184] Where C red is the optimized algorithm complexity, C orig is the original complexity, and R is the complexity reduction ratio; an efficient quantization algorithm is used to reduce computational requirements.

[0185] The formula for adjusting the performance of the inference engine in this step is as follows:

[0186] P adp = f(C,F)

[0187] Where P adp represents the adapted inference performance, C is the cache size, and F is the processor frequency; the inference engine settings are adjusted according to hardware characteristics to adapt to different workloads.

[0188] Through these optimization measures, this patent technology ensures that efficient 4bit quantization model inference can be effectively performed on ARM architecture with limited resources, suitable for real-time data processing and machine learning tasks.

[0189] Step (4) On-chip storage of 4bit model and inference

[0190] First, the on-chip storage encoding of the weights is performed, and then the inference framework is adjusted.

[0191] In order to improve memory utilization and reduce storage requirements, this scheme adopts an efficient weight storage method,

the step (4) is specifically

[0192] For example Figure 2As shown, the compression encoding and decoding process of the quantized weight, after adjusting the inference framework, the weight decoding will be decoded into a new array w_ram[] as the weight of this layer and the output of the activation function of the last layer are input into the convolution operator for convolution operation, and the output obtained after the activation function is used as the activation input of the next layer;

[0193] In the inference process, since the multiplier in the MCU accepts the lowest input of 8 bits, the present scheme designs a decoding module, and in order to not increase the inference occupation of RAM, a layer-by-layer decoding method is adopted, and the inference framework of the present scheme will call the decoding module to convert the model weight from the 4-bit compression format to the 8-bit format required for inference;

[0194] In the inference function, before calling the convolution operator of each layer, the model weight will be prepared by the decoding module before each layer inference, and after the dequantization, the weight in the inference process can be input to the convolution operator in the correct format, and can be correctly accessed and processed. The present scheme adopts a layer-by-layer decoding method, and the weight of each layer can be directly released after operation, and the next layer weight is read into the memory for calculation after release, which increases the operation speed and reduces the additional occupation of memory.

[0195] In addition, in order to cooperate with the above storage scheme, the present application also provides a Python script for storing the generated 4-bit weight in the const unsigned char array according to the above encoding form. This script simplifies the process of converting the quantized 4-bit weight into the 4-bit compression format used in the present application.

[0196] Through the above technical scheme, the present application not only significantly improves the storage and operation efficiency in the tinyML device, but also optimizes the inference framework, which makes it possible to realize complex machine learning models in resource-limited environments. These optimizations ensure the competitiveness of the present application in tinyML applications, and provide a practical and cost-effective solution;

[0197] Step (5) data acquisition of the camera and real-time inference of the data

[0198] First, the spi transmission of the camera data is performed, then the reshape processing of the collected data is performed, and finally the real-time inference of the data is performed;

[0199] In this stage, the camera communicates with the microcontroller through the SPI interface. SPI is a high-speed full-duplex communication protocol suitable for short-distance communication; the advantage of using SPI lies in its simple hardware interface and the ability to support high data transmission rates, which is crucial for real-time monitoring systems. In this patent, both synchronous operation and data frame format are used to collect and process the data collected by the camera.

[0200] Synchronous operation: In SPI communication, the camera acts as a slave device, while the microcontroller acts as the master device; the synchronous transmission of data is controlled by the clock signal provided by the master device.

[0201] Data frame format: Data frames are usually designed to be fixed in length, which can be configured according to the resolution of the camera used and the required image quality; for tinyML systems, the data frame size needs to be carefully designed to ensure that neither the system is overloaded nor the necessary image details are lost.

[0202] To achieve efficient data collection, the Arducam camera is directly connected to the mainboard in this technical solution, and data transmission is achieved through carefully designed jumper wiring. The following is a detailed wiring guide to ensure accurate signal transmission and reliable operation of the system.

[0203] Spi interface configuration:

[0204] Master output slave input (MOSI): Connect the MOSI pin of Arducam to PB15 (D11) of the microcontroller.

[0205] Master input slave output (MISO): Connect the MISO pin of Arducam to PB14 (D12) of the microcontroller.

[0206] Clock line (SCK): Connect the SCK pin of Arducam to PI_1 (D13) of the microcontroller.

[0207] Chip select (CS or NSS): Connect the CS pin of Arducam to PI_0 (D5) of the microcontroller.

[0208] In addition, for power supply and reference ground, VCC is connected to 3.3V, and GND is connected to the ground (GND) on the microcontroller.

[0209] I2C interface configuration:

[0210] Serial clock line (SCL): Connect the SCL pin of Arducam to PB8 (D15) of the microcontroller.

[0211] Serial Data Line (SDA): Connect the SDA pin of the Arducam to PB9 (D14) of the microcontroller.

[0212] With this wiring method, we can achieve high-speed transmission and stable reception of data, thereby supporting the tinyML system to process image data quickly and accurately for real-time monitoring and analysis.

[0213] The reshape processing of the collected data in this step includes the following steps:

[0214] (5-1) Data Buffering: The original image data collected by the camera is first stored in a temporary buffer; this buffer serves as a temporary storage for data, ensuring the integrity and stability of data during processing;

[0215] B(x,y) = I(x,y)

[0216] As shown in the above formula, the data buffering step, where B(x,y) represents the pixel value in the buffer, I(x,y) represents the pixel value at the corresponding position in the original image, and I is the original image data copied to the buffer;

[0217] (5-2) Data Preprocessing: This part of the patent uses denoising, contrast enhancement and normalization, which are completed in the data buffer to avoid additional memory usage;

[0218]

[0219] The above formula is for image data normalization, which maps the pixel value I(x,y) to a unified range, usually [0.1]; here max(I) and min(I) represent the maximum and minimum values in the image data, respectively;

[0220] (5-3) Dimension Conversion: For image data of a monitoring system, dimension conversion involves converting two-dimensional image data into a three-dimensional tensor compatible with model input; this step is achieved through a dimension transformation algorithm that rearranges the original two-dimensional image matrix into the required height, width and channel number for the model; the dimension conversion process is as follows:

[0221] T hwc = reshape(I h w ,(H,W,C));

[0222] Without changing the pixel values, only changing their arrangement in memory; where I h w is the original two-dimensional image data, T h wc is the converted three-dimensional tensor, H, W and C represent the height, width and channel number of the image, respectively;

[0223] Real-time inference of data

[0224] After the reshape processing is completed, the data is ready to be input into the model for real-time inference.

[0225] Inference engine interface: The processed image data is sent to the model through the tinyML inference engine interface described above. This interface is responsible for managing the data flow to the inference core and ensuring that the data is processed in the correct order and format.

[0226] Real-time performance monitoring: During real-time inference, the system continuously monitors the latency and throughput to ensure the timeliness of the inference. The calculated frame rate is printed in real time on the LCD display.

[0227] Output interpretation: The output of the model is then interpreted as useful monitoring information, i.e. whether there is a person, if the result after forward propagation is a person, then print person on the screen, otherwise print no person.

[0228] The above description is only for the implementation of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made by those skilled in the art within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A 4-bit quantization-based intelligent monitoring tinyML method, characterized in that: include: Step (1) Training of the Person Detection Model First, prepare and preprocess the data set, select and customize the person detection model, and then train and verify the person detection model; The selection and customization of the personnel detection model adopts the MobileNetV2 structure to reduce the complexity by using depthwise separable convolution. The formula is as follows: Among them, DConv is depth convolution, PConv is point convolution, X and Y represent input and output features respectively; The output layer of the MobileNetV2 structure was also adjusted, and the final fully connected layer formula was modified to: Among them, FC custom is the adjusted fully connected layer; The training and verification of the personnel detection model uses binary cross entropy as the loss function, and the formula is as follows: in, Is the model for X i The predicted value of The optimizer selects Adam, and the formula of its parameter update rule is as follows: Among them, θ represents the model parameters, η is the learning rate, and are the first-order and second-order moment estimates of the gradient, is a small constant used to prevent division by zero; Step (2) Post-training quantization of the model First, 4-bit linear quantization is performed, and then the activation function is quantized and calibrated, and bias_correction is corrected; The linear transformation formula of the linear quantization is as follows: Where x represents the original high-precision parameter, Q(x) represents the quantized low-precision parameter, ∆ represents the quantization interval, and Z is the quantization zero offset; the quantization operation uses rounding to minimize the quantization error; In the context of 4-bit quantization, the determination of the quantization interval Δ and the quantization zero point Z is crucial; the quantization interval is calculated using the following formula: For 4-bit quantization, n=4, so the denominator is 2 4 -1=15; where max(x) and min(x) represent the maximum and minimum values ​​within the range of data considered for quantization, respectively; After determining the quantization interval, the zero offset Z is set to: The quantization function can be further specified as: ; Step (3) Building the AI ​​inference framework on the MCU First increase the system throughput, then reduce the algorithm complexity, and then adjust the inference engine performance; Step (4) On-chip storage and inference of 4-bit models First, the weights are encoded for on-chip storage, and then the inference framework is adjusted. Specifically, the model weights, which can only be stored in 8-bit units, are encoded into an 8-bit unsigned char type by encoding two adjacent 4-bit weights. After adjusting the inference framework, the weights are decoded into a new array w_ram[] as the weight of this layer. Together with the output of the activation function of the previous layer, it is input into the convolution operator for convolution operation. The output is then passed through the activation function as the activation input of the next layer. During inference, since the minimum input accepted by the multiplier in the MCU is 8 bits, a decoding module is designed to perform decoding in a layer-by-layer manner. The inference framework calls the decoding module to convert the model weights from the 4-bit compressed format to the 8-bit format required for inference. In the inference function, before calling the convolution operator of each layer, the model weights are prepared by the decoding module before each layer of inference. After dequantization, this ensures that the weights are input to the convolution operator in the correct format during inference and can be correctly accessed and processed. Step (5) Camera data collection and real-time data inference First, the camera data is transmitted via SPI, then the collected data is reshaped, and finally the data is inferred in real time.

2. The 4-bit quantization-based intelligent monitoring tinyML method according to claim 1, characterized in that: The training of the person detection model in step (1) also includes the introduction of transfer learning in the training process, by using the pre-trained weights W pre And freeze the weights of the first k layers to achieve this, the formula is as follows: And W k+1:N It is updated during the training process; Finally, in order to prevent overfitting, the regularization term R(W) is introduced into the loss function, and the formula is as follows: Here λ is the regularization coefficient and R(W) is the regularization term, which is used to penalize the size of the model weights to improve the generalization ability of the model.

3. The 4-bit quantization-based intelligent monitoring tinyML method according to claim 1, characterized in that: The linear quantization described in step (2) also includes an inverse quantization operation, and the formula is as follows: Dequantization allows quantized values ​​to be converted back to an approximation of the original floating-point value for use in subsequent calculations.

4. The 4-bit quantization-based intelligent monitoring tinyML method according to claim 3, characterized in that: The quantization calibration of the activation function in step (2) includes acquisition of activation distribution, determination of quantization range and determination of quantization parameters, wherein: The activation distribution collection specifically involves running the forward propagation of the model before quantization to collect the activation function output values ​​of different layers; collecting the statistical distribution of these output values, including their minimum, maximum, mean and standard deviation; these statistics will be used for subsequent quantization parameter determination; The determination of the quantization range is specifically to determine a quantization range according to the statistical distribution of the activation function output value; The quantization parameter is determined by using the determined quantization range to calculate the quantization step size Δ and the zero point Z, as shown in the following formula: Among them, Range is the quantization range determined based on statistical data, and n is the quantization bit width.

5. The 4-bit quantization-based intelligent monitoring tinyML method according to claim 4, characterized in that: In 4-bit quantization, the quantization of weights and activation values ​​often leads to systematic deviations in the model; the deviation correction mechanism includes the mean shift correction coefficient and quantization scale factor ; The mean shift correction coefficient measures the weight W c The actual mean and quantized weight The difference between the means of ; its calculation expression is as follows: in Represents weight W c The expected value or average value of The quantized weight expected value; The quantization scale factor parameter measures the impact of the quantization operation on the weight scale by comparing the norm distribution of the weight before and after quantization. The formula is defined as follows: Here, Represents the original weight W c The L2 norm relative to its mean, It represents the quantized weight L2 norm relative to its mean; The value of is used to adjust the quantization weights so that scale changes during the quantization process can be compensated.

6. The 4-bit quantization-based intelligent monitoring tinyML method according to claim 1, characterized in that: The formula for increasing the system throughput in step (3) is as follows: Among them, T opt represents the optimized system throughput, T base is the baseline throughput, N represents the number of processor cores, and L opt This is the optimized processing latency; by parallelizing operations and reducing single-core processing latency, the overall throughput is increased.

7. The 4-bit quantization-based intelligent monitoring tinyML method according to claim 1, characterized in that: The formula for reducing the algorithm complexity in step (3) is as follows: Among them, C red is the complexity of the optimized algorithm, C orig is the original complexity, R is the complexity reduction ratio; efficient quantization algorithms are used to reduce computational requirements.

8. The 4-bit quantization-based intelligent monitoring tinyML method according to claim 1, characterized in that: The formula for adjusting the inference engine performance described in step (3) is as follows: Among them, P adp represents the adapted inference performance, C is the cache size, and F is the processor frequency. Adjust the inference engine settings based on hardware characteristics to adapt to different workloads.

9. The 4-bit quantization-based intelligent monitoring tinyML method according to claim 1, characterized in that: The step (4) also provides a Python script for storing the generated 4-bit weights in a const unsigned char array according to the above encoding form.

10. The 4-bit quantization-based intelligent monitoring tinyML method according to claim 1, characterized in that: The reshape processing of the collected data in step (5) includes the following steps: (5-1) Data buffering: The raw image data collected by the camera is first stored in a temporary buffer; , as shown in the above formula, is the data buffering step, where B(x,y) represents the pixel value in the buffer, and I(x,y) represents the pixel value at the corresponding position in the original image. This is to copy the original image data to the buffer; (5-2) Data preprocessing: ， The above formula normalizes the image data, mapping the pixel value I(x,y) to a uniform range; here max(I) and min(I) represent the maximum and minimum values ​​in the image data respectively; (5-3) Dimension conversion: For image data of a monitoring system, dimensionality conversion involves converting the two-dimensional image data into a three-dimensional tensor compatible with the model input; this step is achieved through a dimensionality transformation algorithm that rearranges the original two-dimensional image matrix to the height, width, and number of channels required by the model; the dimensionality conversion process is as follows: The pixel values ​​are not changed, only their arrangement in memory is changed; is the original two-dimensional image data, It is the converted three-dimensional tensor, where H, W, and C represent the height, width, and number of channels of the image, respectively.

Images