Demonstration system of deep learning embedded platform

Abstract

The technical scheme of the invention discloses a demonstration system of a deep learning embedded platform. According to the invention, a user can connect different models of embedded AI development boards with the system disclosed by the invention to play a role of similar middleware. And the functions of uploading, compressing, converting, compiling, testing and the like of different artificial intelligence tasks are executed in the system and are finally visualized to a front-end interface, so that a user can conveniently carry out unified operation and execution. Based on demand analysis on the demonstration system, the demonstration system disclosed by the invention has the characteristics that a web page is designed by using a Vue framework, the dynamic display of a working state is supported, and the web page supports visual operation management.

Description

Technical Field

[0001] The present invention relates to a demonstration system for a deep learning embedded platform. Background Art

[0002] With the popularity of the Internet of Things (IoT), embedded systems have become the core components of various smart devices. These systems are widely used in various fields from smart homes to industrial automation, and undertake functions such as device control, data acquisition, communication, and execution of specific tasks. Traditional embedded system designs are often targeted at specific hardware and application scenarios, with the characteristics of limited resources, low power consumption, and high real-time performance. However, with the rapid development of artificial intelligence (AI) technology, the application requirements of embedded systems are evolving from single functions to intelligence and automation, which brings new challenges to embedded systems.

[0003] AI technologies, including deep learning, reinforcement learning, swarm intelligence and autonomous technologies, are widely used. Artificial intelligence mainly refers to machine learning algorithms based on deep learning, including convolutional neural networks, recurrent neural networks, adversarial generative networks, etc. These network models can achieve functions such as target detection and classification, speech recognition, style transfer, etc. Different architectures implemented by processors will lead to differences in cost, power consumption, etc. Since the power consumption and performance of general instruction sets cannot reflect the advantages of the embedded end, major hardware vendors have abandoned general instruction sets and developed their own instruction sets. However, these instruction sets are not universal, which greatly increases the complexity of embedded AI development. At present, the most popular deep learning frameworks, such as TensorFlow, MXNeat, Caffe and PyTorch, support acceleration on limited types of GPU servers. This acceleration depends on the AI ​​acceleration library provided by the vendor. At the same time, models that are wider, deeper, and larger in size tend to have higher accuracy than models that are narrower, shallower, and smaller in size. However, wider, deeper, and larger models require higher computing resources. Now, model applications are increasingly deployed from the cloud to the edge. Due to the limitations of the computing resources of edge devices, we have to consider the device design architecture, storage space (storage), device memory size (memory), device operating power consumption (power), and latency (latency), etc., especially in resource-constrained edge application scenarios such as mobile terminals and embedded devices, which require early adjustment and optimization of applications.

[0004] However, these AI applications usually require high computing power and a large amount of storage resources, which conflicts with the resource limitations of embedded systems. To integrate AI into embedded devices, multiple technical challenges need to be solved, such as efficient use of computing resources, optimization of power consumption, and meeting real-time requirements. At this time, embedded AI application platforms came into being and became an important bridge connecting AI technology and embedded hardware. Embedded AI application platforms integrate a series of optimized tool chains, lightweight AI models, and hardware acceleration technologies to enable AI algorithms to run efficiently on resource-constrained embedded hardware. These platforms usually include the following key components:

[0005] 1. Lightweight AI inference engine: In order to efficiently run AI models on embedded devices, application platforms usually use specially designed lightweight inference engines, such as TensorFlow Lite, ONNX Runtime, ARM Compute Library, etc. These engines are highly optimized to perform complex AI inference tasks on limited computing resources, and significantly reduce computing and memory overhead while ensuring accuracy.

[0006] 2. Hardware acceleration support: Embedded AI application platforms usually support the use of hardware accelerators (such as GPU, NPU, DSP, etc.) to improve the execution efficiency of AI algorithms. Through hardware acceleration, the speed of AI reasoning is greatly improved, while reducing the CPU load and overall power consumption. This is particularly critical for embedded devices that need to handle complex AI tasks, and can extend the battery life of the device while ensuring performance.

[0007] 3. Model compression and optimization technology: Before deploying AI models to embedded devices, the application platform will use various model compression technologies, such as quantization, pruning, and knowledge distillation, to reduce the size and computational complexity of the model. These technologies enable large-scale AI models to be executed efficiently in embedded environments, creating conditions for the popularization of smart devices.

[0008] 4. Cross-platform support and compatibility: An efficient embedded AI application platform needs to support multiple embedded hardware architectures (such as ARM, RISC-V, x86, etc.) and be able to be seamlessly integrated into different operating systems (such as FreeRTOS, Linux, Zephyr, etc.). This cross-platform capability allows developers to easily deploy AI functions on a variety of devices, thereby accelerating the product's time to market.

[0009] 5. Through the embedded AI application platform, developers can quickly integrate AI functions on different embedded hardware to meet the needs of various intelligent applications. For example, in smart home devices, the AI ​​platform can realize voice recognition and face detection functions; in the industrial Internet of Things, the platform can be used for fault prediction and automatic control. These AI applications not only improve the intelligence level of equipment, but also greatly expand the application scenarios of embedded systems, and promote the deep integration and popularization of intelligent technology in various industries. Summary of the invention

[0010] The purpose of this invention is to build a visualization system that can be used to connect deep learning algorithms on different embedded hardware platforms.

[0011] In order to achieve the above object, the technical solution of the present invention is to disclose a demonstration system of a deep learning embedded platform, which is characterized by comprising:

[0012] Virtualize different hardware platforms into a partition in the system, and connect the hardware platforms to the demonstration system by adjusting ping and setting corresponding IP modes;

[0013] Classify deep learning algorithms and adapt them to corresponding hardware platforms;

[0014] Transmit, add, delete, modify, find commands between the client and server, and upload to the server, in addition to using basic languages ​​including Vue, CSS, and JavaScript, and documents used for connection transmission on the server side, use Ajax to update parts of the web page;

[0015] The front-end framework of Vue3.0 is used and developed on Vscode.

[0016] Preferably, when new hardware is added, a new connection is automatically generated through the expansion method in the demonstration system, and after the hardware information is added to the corresponding database, the newly added hardware is virtualized into the system, and then the algorithm is adapted to the corresponding hardware, thereby realizing a complete virtualization process.

[0017] Preferably, for the Yolov3 algorithm in the deep learning algorithm, the Darknet-53 network is used as the backbone feature extraction network, and the idea of ​​the RestNet residual network structure is combined, and the residual block inside the Yolov3 algorithm uses a jump connection, which promotes the fusion learning of multiple different features.

[0018] Preferably, Axios is used for information interaction between the front-end and back-end. In front-end development, Axios is used to communicate with the back-end server to obtain and send data, thereby achieving efficient interaction with the server.

[0019] The present invention allows users to connect different types of embedded AI development boards to the system disclosed by the present invention, playing the role of middleware. The system can also perform functions such as uploading, compressing, converting, compiling and testing different artificial intelligence tasks, and finally visualize them in the front-end interface, so that users can perform unified operations and executions.

[0020] Based on the demand analysis of the demonstration system, the demonstration system disclosed in the present invention has the following features: the web page is designed using the Vue framework, the working status is dynamically displayed, the web page supports visual operation management, and the following technical problems are mainly solved:

[0021] 1. The target machine for testing and the host machine for development used to adapt the deep learning embedded platform algorithm to the general middleware have the Ubuntu operating system and the Linux kernel, and are responsible for managing the system's software and hardware resources;

[0022] 2. The general middleware for deep learning embedded platform algorithm adaptation has a comprehensive data function library, realizes the decoupling of software and hardware, and shields the application layer from the processing details of the hardware;

[0023] 3. The general middleware used for deep learning embedded platform algorithm adaptation has the ability of operator optimization;

[0024] 4. The general middleware for deep learning embedded platform algorithm adaptation has a visual interface to display the operation status of algorithm deployment and the deployment results;

[0025] 5. The general middleware used for deep learning embedded platform algorithm adaptation can realize the deployment of 1 to 3 mainstream AI algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 The working process of the system disclosed in the present invention is illustrated. DETAILED DESCRIPTION

[0027] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0028] The workflow of the demonstration system disclosed in the embodiment of the present invention includes the following steps:

[0029] 1. Hardware type selection

[0030] 2. Environmental preparation and environmental testing

[0031] 3. Model upload

[0032] 4. Model conversion and operator optimization

[0033] 5. Deployment status display

[0034] Figure 1 The workflow of the entire system is given: first select the hardware type, set the environment corresponding to the selected hardware, further test the set environment, and then upload the algorithm model to the specified folder to prepare for the model conversion operation. Finally, convert the uploaded model and compile it into a low-level executable file to complete the conversion process.

[0035] Based on this background, this demonstration platform integrates software and hardware, which can shorten the algorithm deployment cycle on different hardware and promote the development of universal end-side algorithm deployment technology. By connecting different hardware in the virtual machine, different demonstration interfaces can be displayed respectively, thus achieving the function of middleware. Currently, debugging has been carried out on the algorithm A page, algorithm B page, algorithm C page and the corresponding hardware devices.

[0036] The traditional demonstration platform is developed by Java technology, which has poor compatibility with AI projects, and the design effect is relatively old compared to Vue, so Vue is selected as the front-end design framework. Vue.js adopts the principle of responsive design, which can easily manage and update the status in the view layer, realize data-driven UI updates, and provide a smoother user experience.

[0037] The specific steps are as follows:

[0038] 1. Classify different hardware and virtualize the hardware

[0039] First, different hardware platforms are virtualized into a partition in the system, and the hardware platform and the system are connected by adjusting the ping and setting the corresponding IP method. If new hardware is added later, new connections can be automatically generated through the expansion method in the system. Simply add the hardware information to the corresponding database to virtualize the new hardware into the system. The algorithm can be adapted to the corresponding hardware later, thus realizing the complete virtualization process.

[0040] 2. Classify Deep Learning Algorithms

[0041] There are two obvious advantages over the Yolov3 algorithm: the first advantage is that it uses the Darknet-53 network as the backbone feature extraction network and combines the idea of ​​the RestNet residual network structure. The convolution layer mainly includes two types of filters, 1×1 and 3×3. The former filter is used to compress features, and the latter filter is mainly used to expand the number of channels by reducing width and height. One of the biggest features of this structure is that it can improve accuracy by increasing the corresponding depth, but it will also cause gradient explosion and gradient disappearance problems in the training model. The residual block inside the Yolov3 algorithm uses jump connections to promote the fusion learning of multiple different features.

[0042] 3. Use Ajax for server transmission

[0043] In the embodiment of the present invention, it is necessary to transmit, add, delete, modify, and search commands between the client and the server, and upload them to the server. In addition to using basic languages ​​including Vue, CSS, and JavaScript, and documents used for connection transmission on the server side, the embodiment of the present invention uses Ajax to update part of the web page, which is characterized by the redisplay of data without reloading the entire page. Therefore, Ajax (asynchronous JavaScript and XML) can be used as a suitable technology to meet the needs of exchanging small amounts of data between servers. It allows testers and customers to send and test commands more intuitively. Axios is used for information interaction between the front and back ends. Axios is a modern HTTP request library based on Promise for browsers and Node.js environments. It provides a concise and powerful API that makes it easier and more flexible to process HTTP requests in front-end projects. One of the design goals of Axios is to make sending HTTP requests simple and reliable. It is based on Promise, allowing us to process requests using asynchronous programming, and the syntax is concise and easy to understand. In front-end development, through Axios, we can more easily communicate with the back-end server, obtain data, send data, and achieve efficient interaction with the server.

[0044] 4. Front-end development uses the Vue 3.0 framework

[0045] The embodiment of the present invention uses the front-end framework of Vue3.0 and is developed on Vscode. The reason for using this technology is that the front-end updates too fast. The currently used Vue.js 3.0 is a major update, which provides front-end developers with more powerful and flexible tools to build modern and efficient user interfaces. At the same time, Vue 3.0 uses a new responsive system, which provides faster rendering speed and lower memory usage. The optimization of this system is particularly obvious for processing large-scale data changes. Vue 3.0 has carried out many aspects of performance optimization, including smaller size and faster loading time. This makes Vue 3.0 perform better on mobile terminals and low-bandwidth environments. Vue 3.0 provides better support for TypeScript, including more powerful type deduction and more friendly type error prompts. This provides more powerful tool support for the development of large projects.

Claims

1. A demonstration system for a deep learning embedded platform, characterized in that: include: Virtualize different hardware platforms into a partition in the system, and connect the hardware platforms to the demonstration system by adjusting ping and setting corresponding IP modes; Classify deep learning algorithms and adapt them to corresponding hardware platforms; Transmit, add, delete, modify, find commands between the client and server, and upload to the server, in addition to using basic languages ​​including Vue, CSS, and JavaScript, and documents used for connection transmission on the server side, use Ajax to update parts of the web page; The front-end framework of Vue3.0 is used and developed on Vscode.

2. A demonstration system for a deep learning embedded platform as claimed in claim 1, characterized in that: When new hardware is added, a new connection is automatically generated through the expansion method in the demonstration system. After the hardware information is added to the corresponding database, the newly added hardware is virtualized into the system, and the algorithm is subsequently adapted to the corresponding hardware, thereby realizing a complete virtualization process.

3. A demonstration system for a deep learning embedded platform as claimed in claim 1, characterized in that: For the Yolov3 algorithm in the deep learning algorithm, the Darknet-53 network is used as the backbone feature extraction network, and the idea of ​​the RestNet residual network structure is combined. The residual block inside the Yolov3 algorithm uses jump connections, which promotes the fusion learning of multiple different features.

4. A demonstration system for a deep learning embedded platform as claimed in claim 1, characterized in that: Axios is used to exchange information between the front-end and back-end. In front-end development, Axios is used to communicate with the back-end server to obtain and send data, thereby achieving efficient interaction with the server.

Images