A micro hardware machine learning method and system

Abstract

The present invention relates to the field of machine learning technology, and specifically to a micro hardware machine learning method and system, including: optimizing the TensorFlow Lite runtime library based on work scenarios; matching hardware acceleration modules based on work scenarios; accelerating the calculation of machine learning tasks based on the computing power of hardware acceleration modules and multi-core processors, thereby optimizing the TensorFlow Lite runtime library and integrating hardware acceleration modules to achieve a high-performance, low-power machine learning algorithm based on MCU. While maintaining low power consumption, the algorithm can meet the needs of edge devices to achieve local intelligent computing, and is suitable for the Internet of Things, edge computing and other fields.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a micro hardware machine learning method and system. Background Art

[0002] With the continuous development of artificial intelligence and the Internet of Things, the demand for local intelligent computing on edge devices is increasing. TensorFlow Lite is a lightweight machine learning framework launched by Google, suitable for mobile and edge devices, for the development and deployment of various machine learning and deep learning models. TensorFlow has a rich ecosystem, including a large number of pre-trained models, open source projects, and tutorials. Users can use these resources to quickly learn and practice machine learning and deep learning techniques. However, deploying TensorFlow Lite on resource-constrained MCUs still faces challenges in terms of performance, power consumption, and storage space. Summary of the invention

[0003] The purpose of the present invention is to provide a micro-hardware machine learning method and system, which aims to meet the needs of edge devices for local intelligent computing while maintaining low power consumption.

[0004] To achieve the above-mentioned object, in a first aspect, the present invention provides a micro hardware machine learning method, comprising optimizing the TensorFlow Lite runtime library based on a working scenario;

[0005] Match hardware acceleration modules based on work scenarios;

[0006] Accelerate machine learning tasks based on the computing power of hardware acceleration modules and multi-core processors.

[0007] The specific steps of the TensorFlow Lite runtime library optimized based on the working scenario include:

[0008] Determine the working scenario and get the basic model based on the runtime library;

[0009] Identify redundancies in the base model and process them into simple computational layers;

[0010] Quantify the model;

[0011] Compress the model to obtain the optimized TensorFlow Lite runtime library.

[0012] The specific steps of quantizing the model include:

[0013] Prepare input and output data and convert them into a format that matches TensorFlow;

[0014] The original data set is divided into a training set and a validation set. The training set is used to train the model, while the validation set is used to measure the model performance.

[0015] Build models using the machine learning algorithm of your choice and tune hyperparameters to optimize model performance;

[0016] Train the model using the training set data;

[0017] Test the model using the validation set;

[0018] Convert model weights and biases to lower precision data types;

[0019] Apply the quantized model to the test set.

[0020] The specific steps of compressing the model to obtain an optimized TensorFlow Lite runtime library include:

[0021] Train a large neural network to achieve optimal performance;

[0022] Evaluate each parameter and remove unnecessary parameters and those that do not affect the final performance;

[0023] Use regularization to avoid overfitting and further optimize the model;

[0024] Test the model on the target device.

[0025] The specific steps of matching the hardware acceleration module based on the working scenario include:

[0026] Check the hardware accelerators supported by TensorFlow Lite;

[0027] Configure the TensorFlow Lite runtime library;

[0028] Integrated hardware driver library;

[0029] After completing the configuration and integration of the hardware accelerator, conduct testing and performance optimization.

[0030] The specific steps of accelerating the calculation of a specific machine learning task based on the computing power of the hardware acceleration module and the multi-core processor include:

[0031] Clean and preprocess the data;

[0032] Carry out data loading and strategy division;

[0033] Compute machine learning tasks based on hardware acceleration modules and models.

[0034] After calculating the machine learning task based on the hardware acceleration module and the model, the method further includes retrieving idle devices and distributing the task to the idle devices for calculation.

[0035] In a second aspect, the present invention further provides a micro hardware machine learning system, comprising: an MCU, a runtime module, a hardware acceleration module, a memory, a communication interface, and a power management module, wherein the MCU is used to perform machine learning tasks;

[0036] The runtime module is used to provide modules related to machine learning;

[0037] The hardware acceleration module is used to accelerate the calculation of machine learning tasks;

[0038] The memory is a flash memory for storing machine learning models, programs and data;

[0039] The communication interface is used to connect to an external device;

[0040] The power management module is used to optimize the system power consumption.

[0041] Wherein, the communication interface includes UART, I2C, SPI, and CAN.

[0042] Wherein, the memory includes flash memory and RAM.

[0043] A micro hardware machine learning method and system of the present invention, a micro hardware machine learning method of the present invention, including: optimizing the TensorFlow Lite runtime library based on the working scenario; matching the hardware acceleration module based on the working scenario; accelerating the calculation of the machine learning task based on the computing power of the hardware acceleration module and the multi-core processor, so that the high-performance, low-power machine learning algorithm based on the MCU can be realized by optimizing the TensorFlow Lite runtime library and integrating the hardware acceleration module. While maintaining low power consumption, the algorithm can meet the needs of edge devices to realize local intelligent computing, and is suitable for the fields of the Internet of Things, edge computing, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 It is a flowchart of a micro hardware machine learning method according to the first embodiment of the present invention.

[0046] Figure 2 This is a flowchart of optimizing the TensorFlow Lite runtime library based on a work scenario according to the first embodiment of the present invention.

[0047] Figure 3 It is a flowchart of quantizing a model according to the first embodiment of the present invention.

[0048] Figure 4 This is a flowchart of the first embodiment of the present invention for compressing the model to obtain an optimized TensorFlow Lite runtime library.

[0049] Figure 5 It is a flowchart of a hardware acceleration module based on work scenario matching according to the first embodiment of the present invention.

[0050] Figure 6 This is a flowchart of the first embodiment of the present invention for accelerating the calculation of machine learning tasks based on the computing power of a hardware acceleration module and a multi-core processor. DETAILED DESCRIPTION

[0051] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0052] First embodiment

[0053] See also Figure 1 to Figure 6 , Figure 1 It is a flowchart of a micro hardware machine learning method according to the first embodiment of the present invention. Figure 2 This is a flowchart of optimizing the TensorFlow Lite runtime library based on a work scenario according to the first embodiment of the present invention. Figure 3 It is a flowchart of quantizing a model according to the first embodiment of the present invention. Figure 4 This is a flowchart of the first embodiment of the present invention for compressing the model to obtain an optimized TensorFlow Lite runtime library. Figure 5 It is a flowchart of a hardware acceleration module based on work scenario matching according to the first embodiment of the present invention. Figure 6 This is a flowchart of the first embodiment of the present invention for accelerating the calculation of machine learning tasks based on the computing power of a hardware acceleration module and a multi-core processor.

[0054] The present invention provides a micro hardware machine learning method, comprising:

[0055] S101 optimizes the TensorFlow Lite runtime library based on work scenarios;

[0056] The TensorFlow Lite runtime library is one of the core components of TensorFlow Lite. It contains the basic functions and modules necessary to run machine learning models on different hardware platforms. The library can help developers quickly deploy and run models and can be used without any additional installation. The specific steps include:

[0057] S201 determines the working scenario and obtains the basic model based on the TensorFlow Lite runtime library;

[0058] Identify the specific work scenario in which you want to apply the TensorFlow Lite model. For example, image classification, object detection, speech recognition, etc. Then, collect and prepare training datasets related to the target scenario. These datasets should contain correctly labeled samples for training and evaluating the model. Then select a neural network model suitable for the target scenario and build the model structure in the TensorFlow framework. You can use the high-level API (such as Keras) or low-level API provided by TensorFlow to customize the model.

[0059] S202 identifies redundant parts in the basic model and processes them into simple calculation layers;

[0060] In the process of building a machine learning model, some redundant parts are often generated, such as extra layers or excessive model size. In order to make the model more concise and efficient, first of all, we need to conduct a comprehensive analysis of the structure of the model to determine which parts may be redundant. This can be done by looking at the model diagram or checking the output and number of parameters of each layer. After confirming which parts are redundant, we can directly delete these redundant layers. For example, for some deep learning models, there may be some optional layers that will not affect the final result at all. In this case, they can be deleted directly to reduce the size of the model. Although some computing layers are useful, they may require a lot of computing resources, which will increase the running time and memory consumption of the model. In this case, you can try to replace these complex computing layers with simpler computing layers, such as fully connected layers or convolutional layers.

[0061] S203 performs quantization processing on the model;

[0062] The specific steps include:

[0063] S301 prepares input and output data and converts them into a format that matches TensorFlow;

[0064] Convert the trained model to TensorFlow Lite format. This can be done by using TensorFlow's tf.lite.TFLiteConverter class.

[0065] S302 divides the original data set into a training set and a validation set, the training set is used to train the model, and the validation set is used to measure the model performance;

[0066] Next, you need to prepare a representative dataset to facilitate calibration during the model quantization process. This dataset should be a part of the entire dataset and it should be able to reflect the performance of the model in a real environment.

[0067] S303 builds a model using the selected machine learning algorithm and adjusts hyperparameters to optimize model performance;

[0068] S304 trains the model using the training set data, and tests the model using the validation set;

[0069] S305 converts the model weights and bias items into lower precision data types;

[0070] S306 applies the quantized model to the test set.

[0071] S204 compresses the model to obtain an optimized TensorFlow Lite runtime library.

[0072] The specific steps include:

[0073] S401 evaluates each parameter and removes non-essential parameters and those that will not affect the final performance;

[0074] S402 uses regularization to avoid overfitting and further optimize the model;

[0075] Regularization is a commonly used technique to prevent overfitting. It can effectively control the complexity of the model and help improve the generalization ability of the model. The following are several common ways to use regularization:

[0076] L2 norm penalty: L2 norm penalty is the most commonly used regularization technique, which can effectively limit the size of the weight matrix to prevent overfitting. The degree of penalty can be adjusted by setting the hyperparameter λ.

[0077] Dropout: Dropout is a random deactivation mechanism that prevents neurons from being too closely connected and reduces the complexity of the model. The deactivation ratio can be controlled by setting the hyperparameter p.

[0078] Regularization term: Adding regularization terms, such as L1 regularization and L2 regularization, to the loss function can effectively control the complexity of the model.

[0079] Weight decay: During the back-propagation process, the weights are decayed to control their growth rate and thus prevent overfitting.

[0080] S403 tests the model on the target device.

[0081] The matching effect of the model can be detected through testing.

[0082] S102 matches the hardware acceleration module based on the working scenario;

[0083] The specific steps include:

[0084] S501 checks the hardware accelerators supported by TensorFlow Lite;

[0085] S502 configures the TensorFlow Lite runtime library;

[0086] S503 integrated hardware driver library;

[0087] S504 performs testing and performance optimization after completing configuration and integration of the hardware accelerator.

[0088] The above method can better match the corresponding hardware accelerator, so that the runtime library can better match the hardware accelerator and improve the performance optimization effect. TensorFlow Lite supports a variety of hardware accelerators, such as CPU, GPU, DSP, and dedicated AI chips. TensorFlow Lite runs on the CPU by default. It uses SIMD instruction set and hardware accelerators to increase computing speed. If the device supports CUDA, you can use the GPU accelerator to increase the inference speed of the model. Please note that this requires the installation of the corresponding driver and a version of TensorFlow that supports CUDA. Some devices also support digital signal processors (DSPs), which are hardware accelerators designed specifically for processing discrete signals. It can be used to accelerate operations such as convolution and tensor operations. For specific tasks, some devices also support dedicated AI chips, such as the TensorFlowLite Micro framework supports the Movidius Myriad X AI chip. The Neural Network API (NNAPI) is a standard API for mobile devices that automatically selects the most appropriate hardware accelerator to increase the speed of model inference.

[0089] S103 accelerates the calculation of machine learning tasks based on the computing power of hardware acceleration modules and multi-core processors.

[0090] The specific steps include:

[0091] S601 cleans and preprocesses the data;

[0092] Cleaning and preprocessing the data before training can make the model better at handling large-scale data sets. For example, you can segment text data and remove irrelevant words; you can downsample and standardize image data.

[0093] S602 performs data loading and strategy division;

[0094] The above methods can effectively reduce the memory required for training. For example, you can use small batch training methods to load only a small part of the data for training each time; or use distributed training to spread the training tasks to multiple nodes.

[0095] S603 computes machine learning tasks based on hardware acceleration modules and models;

[0096] S604 retrieves idle devices and distributes tasks to the idle devices for calculation.

[0097] For very large models and datasets, you can consider using a distributed computing framework for computation and storage, which can distribute tasks across multiple machines or nodes, thereby reducing the memory pressure on a single machine.

[0098] A micro hardware machine learning method of the present invention includes: optimizing the TensorFlow Lite runtime library based on the working scenario; matching the hardware acceleration module based on the working scenario; accelerating the calculation of the machine learning task based on the computing power of the hardware acceleration module and the multi-core processor, so that the high-performance and low-power machine learning algorithm based on the MCU can be realized by optimizing the TensorFlow Lite runtime library and integrating the hardware acceleration module. The algorithm can meet the needs of edge devices to realize local intelligent computing while maintaining low power consumption, and is suitable for the fields of the Internet of Things, edge computing, etc.

[0099] Second embodiment

[0100] Based on the first embodiment, the present invention also provides a micro hardware machine learning system, including: an MCU, a runtime module, a hardware acceleration module, a memory, a communication interface and a power management module, wherein the MCU is used to execute machine learning tasks; the runtime module is used to provide modules related to machine learning; the hardware acceleration module is used to accelerate the calculation of machine learning tasks; the memory is a flash memory used to store machine learning models, programs and data; the communication interface is used to connect to external devices; and the power management module is used to optimize system power consumption.

[0101] In this embodiment, the communication interface includes UART, I2C, SPI, and CAN. The memory includes flash memory and RAM. The memory can store various data for machine learning operation, access corresponding modules through the runtime module, improve computing power through the hardware acceleration module, and connect to external interfaces through the communication interface for expansion, so as to make it more convenient to use.

[0102] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A miniature hardware machine learning approach, It is characterized in that Includes: Optimizing the TensorFlow Lite runtime library based on the work scenario. The specific steps include: Determine the working scenario and get the basic model based on the runtime library; Identify redundancies in the base model and process them into simple computational layers; The model is quantified. The specific steps include: Prepare input and output data and convert them into a format that matches TensorFlow; The original data set is divided into a training set and a validation set. The training set is used to train the model, while the validation set is used to measure the model performance. Build models using the machine learning algorithm of your choice and tune hyperparameters to optimize model performance; Train the model using the training set data; Test the model using the validation set; Convert model weights and biases to lower precision data types; Apply the quantized model to the test set; Compress the model to get the optimized TensorFlow Lite runtime library; Match hardware acceleration modules based on work scenarios; Accelerate machine learning tasks based on the computing power of hardware acceleration modules and multi-core processors.

2. A micro hardware machine learning method as claimed in claim 1, characterized in that: The specific steps of compressing the model to obtain an optimized TensorFlow Lite runtime library include: Evaluate each parameter and remove unnecessary parameters and those that do not affect the final performance; Use regularization to avoid overfitting and further optimize the model; Test the model on the target device.

3. A micro hardware machine learning method as claimed in claim 2, characterized in that: The specific steps of matching the hardware acceleration module based on the working scenario include: Check the hardware accelerators supported by TensorFlow Lite; Configure the TensorFlow Lite runtime library; Integrated hardware driver library; After completing the configuration and integration of the hardware accelerator, conduct testing and performance optimization.

4. A micro hardware machine learning method as claimed in claim 3, characterized in that: The specific steps of accelerating the calculation of a specific machine learning task based on the computing power of the hardware acceleration module and the multi-core processor include: Clean and preprocess the data; Carry out data loading and strategy division; Compute machine learning tasks based on hardware acceleration modules and models.

5. A micro hardware machine learning method as claimed in claim 4, characterized in that: After calculating the machine learning task based on the hardware acceleration module and the model, the method also includes retrieving idle devices and distributing the task to the idle devices for calculation.

6. A micro hardware machine learning system, applied to the micro hardware machine learning method according to claim 1, It is characterized in that It includes: an MCU, a runtime module, a hardware acceleration module, a memory, a communication interface, and a power management module, wherein the MCU is used to perform machine learning tasks; The runtime module is used to provide modules related to machine learning; The hardware acceleration module is used to accelerate the calculation of machine learning tasks; The memory is a flash memory for storing machine learning models, programs and data; The communication interface is used to connect to an external device; The power management module is used to optimize the system power consumption.

7. A micro hardware machine learning system as claimed in claim 6, characterized in that: The communication interfaces include UART, I2C, SPI, and CAN.

8. A micro hardware machine learning system as claimed in claim 7, characterized in that: The memory includes flash memory and RAM.

Images