Deep learning model deployment method and device for arm processor

Abstract

The application provides a deep learning model deployment method and device for an ARM processor. The method comprises the following steps: step 1, installing Docker Desktop on a local electronic device; step 2, cross-compiling a deep learning inference engine for the ARM processor; step 3, training a deep learning model for a specified task and exporting the trained deep learning model; step 4, constructing a deep learning model image for the specified task; step 5, installing Docker on an embedded device; and step 6, pulling the previously trained deep learning model from a public image warehouse Docker Hub and deploying and performing inference acceleration on the trained deep learning model on the embedded device. The application reduces the difficulty of model deployment and operation and maintenance, greatly improves the speed of model deployment and application in an edge computing environment, and provides important technical support for the rapid transformation and landing application of deep learning technology.

Description

TECHNICAL FIELD

[0001] The embodiment of the application relates to the technical field of artificial intelligence, and particularly relates to a deep learning model deployment method and equipment for an ARM processor. BACKGROUND

[0002] Compared with a traditional machine learning model, a deep learning model has achieved satisfactory performance in many fields such as machine vision and natural language processing due to its strong representation ability. The training and deployment of the deep learning model need to build a complete set of complex and tedious basic environment, and the isolation sandbox mechanism of the container technology can ensure that the model is deployed in an independent running environment and does not depend on a specific programming language, framework or system. Different containers can provide different microservices, and it is relatively convenient to call and communicate between the microservices, thereby greatly improving the flexibility of model deployment. However, the local deployment mode cannot realize online rapid and automatic deployment. With the rise of edge computing technology and the popularity of a large number of embedded devices, some researches build a deep learning model basic running environment image based on the container technology, but the image still cannot be applied to embedded devices based on the ARM architecture. Therefore, developing a deep learning model deployment method and equipment for an ARM processor can effectively overcome the defects in the related art, and has become a technical problem to be solved in the industry. SUMMARY

[0003] In view of the above problems existing in the prior art, the embodiment of the application provides a deep learning model deployment method and equipment for an ARM processor.

[0004] In the first aspect, the embodiment of the application provides a deep learning model deployment method for an ARM processor, which comprises the following steps: step 1, installing Docker Desktop on a local electronic device; step 2, cross-compiling a deep learning inference engine for an ARM processor; step 3, training a deep learning model for a specified task and exporting the trained deep learning model; step 4, constructing a deep learning model image for the specified task; step 5, installing Docker on an embedded device; and step 6, pulling the previously trained deep learning model in a public image warehouse Docker Hub and deploying and performing inference acceleration on the trained deep learning model on the embedded device.

[0005] On the basis of the above method embodiment content, the ARM processor-oriented deep learning model deployment method provided in the embodiment of the application, step 1 specifically comprises: installing Docker Desktop on a local Windows computer, and starting WSL in the Windows system, running a Linux kernel in a Windows virtual machine through a virtualization technology, and enabling cross-compilation in the virtual environment of DockerDesktop to realize the compilation of an executable image file for an ARM processor on an AMD processor.

[0006] On the basis of the above method embodiment content, the ARM processor-oriented deep learning model deployment method provided in the embodiment of the application, step 2 specifically comprises: under the cross-compilation environment started in step 1, constructing a deep learning inference engine image for an ARM processor in a cross-compilation manner according to open-source OpenVINO source code and a Dockerfile file thereof; building a basic running environment for a plurality of mainstream deep learning frameworks and classic deep learning models in the engine image, providing support for environment dependencies such as an Ubuntu operating system, a Python programming language, an OpenCV computer vision framework, a Pytorch and a Tensorflow deep learning framework, and pushing the engine image to a public image warehouse DockerHub to realize the rapid acquisition of the image during the deployment of a deep learning model.

[0007] On the basis of the above method embodiment content, the ARM processor-oriented deep learning model deployment method provided in the embodiment of the application, step 3 specifically comprises: preparing data for deep learning model training, and training a deep learning model on a GPU of a local Windows computer based on a mainstream deep learning framework, reading a trained model file and re-exporting it into a model format supported by OpenVINO, including an ONNX model or an IR model of OpenVINO, the IR model being composed of two files with the same name, one being a.bin format file containing network topology information, and the other being an.xml format file containing weight and bias binary data.

[0008] On the basis of the above method embodiment content, the ARM processor-oriented deep learning model deployment method provided in the embodiment of the application specifically comprises the following steps: based on the local Docker Desktop software, a deep learning inference engine image for an ARM processor built in step 2 is pulled from a public image warehouse Docker Hub, and the container internal environment is entered, a deep learning model for a specified task exported in step 3 is run in the basic running environment of the container, and the container running the deep learning model is built into a new image in a cross-compiled manner based on a Docker buildx command, and the image is a deep learning model image for the specified task, which runs in an embedded device system environment with an ARM processor.

[0009] On the basis of the above method embodiment content, the ARM processor-oriented deep learning model deployment method provided in the embodiment of the application specifically comprises the following steps: an embedded device with an ARM processor is prepared, a suitable operating system is configured for the device, the basic components required by Docker are installed and a GPG key is added, and a Docker version supported by the current system is installed according to the specific situation of the operating system.

[0010] On the basis of the above method embodiment content, the ARM processor-oriented deep learning model deployment method provided in the embodiment of the application specifically comprises the following steps: in the embedded device for model deployment, a deep learning model image for a specified task built in step 4 is pulled from a public image warehouse Docker Hub, a Docker start command is executed on the image to enter the container environment for model inference, a neural computing stick is inserted into the USB port of the embedded device for accelerating model inference, specified data is input into the model for inference prediction, and the ARM processor-oriented deep learning model deployment is completed.

[0011] In the second aspect, the embodiment of the application provides an ARM processor-oriented deep learning model deployment device, which comprises: a first main module for realizing step 1, installing Docker Desktop on a local electronic device; step 2, cross-compiling a deep learning inference engine for an ARM processor; a second main module for realizing step 3, training a deep learning model for a specified task and exporting the trained deep learning model; a third main module for realizing step 4, building a deep learning model image for the specified task; step 5, installing Docker on an embedded device; and a fourth main module for realizing step 6, pulling a previously trained deep learning model from a public image warehouse Docker Hub and deploying and inferring the trained deep learning model on the embedded device.

[0012] In a third aspect, embodiments of the present application provide an electronic device, comprising:

[0013] at least one processor; and

[0014] at least one memory connected with the processor, wherein:

[0015] The memory stores program instructions executable by the processor, and the processor invoking the program instructions can execute the ARM processor-oriented deep learning model deployment method provided in any one of the various implementation manners of the first aspect.

[0016] In a fourth aspect, embodiments of the present application provide a non-transitory computer-readable storage medium storing computer instructions, and the computer instructions cause a computer to execute the ARM processor-oriented deep learning model deployment method provided in any one of the various implementation manners of the first aspect.

[0017] The ARM processor-oriented deep learning model deployment method and device provided by the embodiments of the present application ensure the compatibility of embedded devices for various deep learning models, reduce the difficulty of model deployment and operation and maintenance, greatly improve the speed of model deployment and application in the edge computing environment, and provide important technical support for the rapid transformation and landing application of deep learning technology. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings described below are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0019] Figure 1 The ARM processor-oriented deep learning model deployment method flowchart provided by the embodiments of the present application;

[0020] Figure 2 The ARM processor-oriented deep learning model deployment device structure schematic diagram provided by the embodiments of the present application;

[0021] Figure 3 The entity structure schematic diagram of the electronic device provided by the embodiments of the present application;

[0022] Figure 4 The internal structure diagram of the ARM processor-oriented deep learning model rapid deployment device provided by the embodiments of the present application;

[0023] Figure 5The prediction result map of the computer vision image provided by the embodiment of the present application;

[0024] Figure 6 The prediction result map of the satellite remote sensing image provided by the embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the protection scope of the present application. In addition, the technical features in each embodiment or single embodiment provided by the present application can be combined with each other arbitrarily to form a feasible technical solution, and this combination is not restricted by the order of steps and / or structure mode, but should be based on the realization by those skilled in the art. When the combination of technical solutions appears contradictory or unfeasible, it should be considered that the combination of technical solutions does not exist, and is not within the protection scope of the present application.

[0026] The embodiment of the present application provides a deep learning model deployment method for an ARM processor, referring to Figure 1 The method comprises the following steps: step 1, installing Docker Desktop on a local electronic device; step 2, cross-compiling a deep learning inference engine for an ARM processor; step 3, training a deep learning model for a specified task and exporting the trained deep learning model; step 4, building a deep learning model image for the specified task; step 5, installing Docker on an embedded device; step 6, pulling the previously trained deep learning model in a public image repository Docker Hub, and deploying and reasoning the trained deep learning model on the embedded device.

[0027] Based on the content of the above method embodiment, as an optional embodiment, the deep learning model deployment method for an ARM processor provided in the embodiment of the present application, step 1 specifically comprises: installing DockerDesktop on a local Windows computer, and starting WSL in the Windows system, running a Linux kernel in a Windows virtual machine through virtualization technology, and enabling cross-compilation in the virtual environment of Docker Desktop, so as to compile an executable image file for an ARM processor on an AMD processor.

[0028] Based on the content of the above method embodiment, as an optional embodiment, the method for deploying a deep learning model for an ARM processor provided in the embodiment of the application specifically comprises the following steps: in the cross-compiled environment started in step 1, an image of a deep learning inference engine for an ARM processor is built in a cross-compiled manner according to open-source OpenVINO source code and a Dockerfile file; a basic running environment for a plurality of mainstream deep learning frameworks and classic deep learning models is built in the engine image, support for environment dependencies such as an Ubuntu operating system, a Python programming language, an OpenCV computer vision framework, a Pytorch and a Tensorflow deep learning framework is provided, and the engine image is pushed to a public image warehouse Docker Hub, so that the image can be quickly acquired when a deep learning model is deployed.

[0029] Based on the content of the above method embodiment, as an optional embodiment, the method for deploying a deep learning model for an ARM processor provided in the embodiment of the application specifically comprises the following steps: data used for training a deep learning model is prepared, and a deep learning model is trained on a GPU of a local Windows computer based on a mainstream deep learning framework, a model file trained is read and re-exported into a model format supported by OpenVINO, including an ONNX model or an IR model of OpenVINO, the IR model is composed of two files with the same name, one is a.bin format file containing network topology information, and the other is an.xml format file containing binary data of weights and biases.

[0030] Based on the content of the above method embodiment, as an optional embodiment, the method for deploying a deep learning model for an ARM processor provided in the embodiment of the application specifically comprises the following steps: based on a local Docker Desktop software, a deep learning inference engine image for an ARM processor built in step 2 is pulled from a public image warehouse Docker Hub, and enters an internal environment of a container, a deep learning model for a specified task exported in step 3 is run in the basic running environment of the container, and the container running the deep learning model is built into a new image in a cross-compiled manner based on a Docker buildx command, the image is a deep learning model image for the specified task, and runs in an embedded device system environment with an ARM processor.

[0031] Based on the content of the above method embodiment, as an optional embodiment, the method for deploying a deep learning model for an ARM processor provided in the embodiment of the application specifically comprises the following steps: preparing an embedded device with an ARM processor, configuring a suitable operating system for the device, installing the required basic components of Docker and adding a GPG key for the device, and installing a Docker version supported by the current system according to the specific situation of the operating system.

[0032] Based on the content of the above method embodiment, as an optional embodiment, the method for deploying a deep learning model for an ARM processor provided in the embodiment of the application specifically comprises the following steps: preparing an embedded device with an ARM processor, configuring a suitable operating system for the device, installing the required basic components of Docker and adding a GPG key for the device, and installing a Docker version supported by the current system according to the specific situation of the operating system.

[0033] The method for deploying a deep learning model for an ARM processor provided in the embodiment of the application ensures the compatibility of the embedded device for various deep learning models, reduces the difficulty of model deployment and operation and maintenance, greatly improves the speed of model deployment and application in an edge computing environment, and provides important technical support for the rapid transformation and landing application of deep learning technology.

[0034] In embodiment 1, the rapid deployment of a city fire detection model is implemented on a Raspberry PI 4B device, including the following steps:

[0035] Step 1: Install Docker Desktop on a local computer. Install Docker Desktop software in the default Windows environment of a micro-star GL65 notebook computer, and enable WSL function. Run a Linux kernel in a Windows virtual machine through virtualization technology, create a Linux virtual environment in Docker Desktop, and enable cross-compilation function by running binfmt_misc image to compile executable image files for ARM processors on AMD processors.

[0036] Step 2, cross-compile the deep learning inference engine for ARM processors. Log in to the OpenVINO website to download the open-source OpenVINO source code and the provided Dockerfile file. In the cross-compiled environment started in step 1, use the Docker Buildx command to cross-compile the deep learning inference engine image for ARM processors. This image builds a basic runtime environment for various mainstream deep learning frameworks and classic deep learning models, providing support for armv7 architecture, Ubuntu operating system, Python programming language, OpenCV computer vision framework, Pytorch and Tensorflow deep learning framework, and other environment dependencies. Use the Docker command to push the image to the public image repository Docker Hub, and quickly obtain the image when deploying the deep learning model.

[0037] Step 3, train the urban fire detection model and export. Use the computer vision data (FASDD_CV) in the open-source flame and smoke detection dataset (FASDD) as input data for the deep learning model (YOLOv5s-v2.0), and train the deep learning urban fire detection model on a GPU based on the deep learning framework (Pytorch1.5.1) with an iteration training period of 300 times. Read the trained model file (best.pt) and export it to the ONNX format supported by OpenVINO, and name it best.onnx.

[0038] Step 4, build the urban fire detection model image. Start the Docker Desktop software in the local computer, pull the deep learning inference engine image for ARM processors built in step 2 from Docker Hub, and enter its container internal environment. Copy the exported urban fire detection model best.onnx to the container's basic runtime environment and run it. After the program runs normally, use the Docker buildx command again to build a new image with the container running the urban fire detection model in a cross-compiled manner. This image is a deep learning model image for urban fire detection tasks, which can successfully run in an embedded device system environment with an ARM processor.

[0039] Step 5, install Docker on the embedded device. Prepare a Raspberry Pi 4B device and configure the official released operating system (Raspberry Pi Desktop) for the device, install the required basic components for Docker, including apt-transport-https, software-properties-common, ntp, ntpdate, curl and ca-certificates, and add the GPG key for Docker, according to the specific situation of the operating system, install the current system supported Docker version 19.03.5.

[0040] Step 6, fast model deployment and inference acceleration. On the Raspberry Pi 4B device with Docker software installed, pull the city fire detection model image built in step 4 from Docker Hub. Execute the Docker run command to start the city fire detection model image and enter the container environment for model inference, insert the Intel Neural Compute Stick (NCS2) into the USB port of the embedded device for model inference acceleration, input the computer vision image captured by the city fire monitoring system as test data into the model for inference prediction, and complete the fast deployment of the ARM processor-oriented deep learning model.

[0041] In embodiment 2, the fast deployment of the forest fire monitoring model on the Raspberry Pi 4B device is implemented, including the following steps:

[0042] Steps 1 and 2 use the same operation process as in embodiment 1, and the subsequent steps will directly use the execution results, which will not be described here.

[0043] Step 3, train the forest fire monitoring model and export. The satellite remote sensing data (FASDD_RS) in the open-source flame and smoke detection data set (FASDD) is used as the input data of the deep learning model (YOLOv5s-v2.0), and the deep learning forest fire monitoring model is trained on the GPU based on the deep learning framework (Pytorch1.5.1), and the iteration training period is set to 300 times. Read the trained model file (best.pt) and export it to the ONNX format supported by OpenVINO and name it best.onnx.

[0044] Step 4, build the forest fire monitoring model image. Start the Docker Desktop software in the local computer, pull the deep learning inference engine image for ARM processor built in step 2 from Docker Hub, and enter the container internal environment. Copy the exported forest fire monitoring model best.onnx to the base runtime environment of the container and run it. After the program runs normally, build the container running the forest fire monitoring model into a new image in a cross-compiled manner based on the Docker buildx command again. This image is a deep learning model image for forest fire monitoring tasks, which can successfully run in an embedded device system environment with an ARM processor.

[0045] Step 5, install Docker on the embedded device. Prepare a Raspberry Pi 4B device and configure the official operating system (Raspberry Pi Desktop) for the device. Install the basic components required for Docker, including apt-transport-https, software-properties-common, ntp, ntpdate, curl, and ca-certificates. Add the GPG key for Docker, and according to the specific situation of the operating system, install the current system supported Docker version 19.03.5.

[0046] Step 6, fast model deployment and inference acceleration. On the Raspberry Pi 4B device with Docker software installed, pull the forest fire monitoring model image built in step 4 from Docker Hub. Execute the Docker run command to start the forest fire monitoring model image and enter the container environment for model inference. Insert the Intel Neural Compute Stick (NCS2) into the USB port of the embedded device for model inference acceleration. Fuse the red, green, and blue bands of the Sentinel-2 remote sensing image and convert it to an 8-bit true color image. Input the 8-bit true color image as test data into the model for inference prediction. Complete the entire process of fast deployment of deep learning models for ARM processors.

[0047] Figure 4The internal structure of the ARM processor-oriented deep learning model rapid deployment device is shown, including: an ARM processor, a high-performance four-core ARM processor is used to implement the model, and the above embodiments are all configured with a Raspberry Pi operating system matched with an armv7 architecture; an intelligent (AI) coprocessor, a neural computing stick is a deep learning inference tool based on a USB mode and an independent intelligent (AI) coprocessor; and a memory, the memory is used to store a computer executable program that can run on the ARM processor. When the executable program is executed by the processor, the ARM processor-oriented deep learning model rapid deployment device implements the ARM processor-oriented deep learning model rapid deployment method.

[0048] Table 1 inference speed of the urban fire detection model on different processors

[0049]

[0050]

[0051] Table 1 compares the inference speed of the urban fire detection model generated by the present application on different processors. From Table 1, it can be seen that the high-performance CPU of Intel achieves the highest frame rate of 25 frames per second, basically meeting the real-time detection requirement. Since the ONNX model stores the weight information of the neural network based on a serialized data structure, and the GPU is good at parallel computing, the inference speed of the ONNX model on the GPU is limited to a certain extent. With the aid of NCS2 auxiliary calculation, the cost of Raspberry Pi 4B is extremely low, and a satisfactory detection speed is also achieved, which is only 3-4 times that of CPU and GPU. Moreover, it is worth noting that based on the method, device and storage medium provided by the present application, the deployment of the deep learning model on the embedded device only needs a time cost of minutes, and greatly reduces the difficulty of model deployment, which provides strong support for the rapid landing application of intelligent achievements in the edge computing environment.

[0052] Figure 5 The prediction results of the computer vision image in embodiment 1 are shown. From Figure 5 it can be seen that the urban fire detection model constructed in embodiment 1 of the present application achieves good inference detection effect on Raspberry Pi 4B. In the dark environment of the urban underground passage, the model still accurately frames the outline of the flame and the smoke. The Figure 6 The prediction result graph of the satellite remote sensing image in embodiment 2 of the present application is shown. From the prediction result, it can be seen that for large-scale remote sensing images, the forest fire monitoring model also achieves good detection results, and even for the small target of the flame with a very small area ratio, accurate prediction can be achieved. Embodiments 1 and 2 confirm that the present application provides a stable and reliable inference running environment for the deep learning model, and can fully exert the excellent performance of the deep learning model without reservation.

[0053] The application provides a stable, containerized, quickly accessible and deployable model inference environment for various deep learning models, shortens the deployment time of deep learning models on embedded devices to minutes, and provides an ARM processor-oriented deep learning model inference engine for the inference capability of deep learning models on ARM processors.

[0054] The implementation basis of each embodiment of the application is achieved by programmed processing of a device with processor function. Therefore, in engineering practice, the technical solutions and functions of each embodiment of the application can be packaged into various modules. Based on this actual situation, on the basis of each embodiment described above, an embodiment of the application provides an ARM processor-oriented deep learning model deployment device, which is used to execute the ARM processor-oriented deep learning model deployment method in the method embodiment. Figure 2 The device includes a first main module for implementing step 1, installing Docker Desktop on a local electronic device; step 2, cross-compiling an ARM processor-oriented deep learning inference engine; a second main module for implementing step 3, training a deep learning model for a specified task and exporting the trained deep learning model; a third main module for implementing step 4, constructing a deep learning model image for the specified task; step 5, installing Docker on an embedded device; and a fourth main module for implementing step 6, pulling the previously trained deep learning model in a public image repository Docker Hub and deploying and inferring the trained deep learning model on the embedded device.

[0055] The ARM processor-oriented deep learning model deployment device provided by the embodiment of the application adopts several modules in Figure 2 , ensures the compatibility of embedded devices for various deep learning models, reduces the difficulty of model deployment and operation and maintenance, greatly improves the speed of model deployment and application of achievements in the edge computing environment, and provides important technical support for the rapid transformation and landing application of deep learning technology.

[0056] It should be noted that the device in the device embodiment provided by the application can be used to implement the method in the above method embodiment, and can also be used to implement the method in other method embodiments provided by the application. The difference is only that the corresponding function module is set, and the principle is basically the same as that of the above device embodiment provided by the application. As long as the person skilled in the art improves the device in the above device embodiment by combining technical features to obtain the corresponding technical means and the technical solution composed of these technical means on the premise of ensuring the practicability of the technical solution, the corresponding device embodiment can be obtained to implement the method in other method embodiments. For example:

[0057] Based on the content of the above device embodiment, as an optional embodiment, the deep learning model deployment device for ARM processor provided in the embodiment of the application further includes a first sub-module for implementing step 1, which specifically includes installing Docker Desktop on a local Windows computer and starting WSL in the Windows system, running a Linux kernel in a Windows virtual machine through virtualization technology, enabling cross-compilation in the virtual environment of Docker Desktop, and compiling an executable image file for ARM processor on an AMD processor.

[0058] Based on the content of the above device embodiment, as an optional embodiment, the deep learning model deployment device for ARM processor provided in the embodiment of the application further includes a second sub-module for implementing step 2, which specifically includes constructing a deep learning inference engine image for ARM processor in a cross-compilation manner according to the open source OpenVINO source code and its Dockerfile file under the cross-compilation environment started in step 1; building a basic running environment for a plurality of mainstream deep learning frameworks and classic deep learning models in the engine image, providing support for environment dependencies such as Ubuntu operating system, Python programming language, OpenCV computer vision framework, Pytorch and Tensorflow deep learning framework, and pushing the engine image to a public image repository Docker Hub to realize fast acquisition of the image during deep learning model deployment.

[0059] Based on the content of the above device embodiment, as an optional embodiment, the ARM processor-oriented deep learning model deployment device provided in the embodiment of the application further includes: a third sub-module, configured to implement step 3 and specifically configured to: prepare data for deep learning model training, train a deep learning model on a GPU of a local Windows computer based on a mainstream deep learning framework, read the trained model file and re-export it as a model format supported by OpenVINO, including an ONNX model or an IR model of OpenVINO, and the IR model is composed of two files with the same name, one is a.bin format file containing network topology information, and the other is an.xml format file containing weight and bias binary data.

[0060] Based on the content of the above device embodiment, as an optional embodiment, the ARM processor-oriented deep learning model deployment device provided in the embodiment of the application further includes: a fourth sub-module, configured to implement step 4 and specifically configured to: based on a local Docker Desktop software, pull the ARM processor-oriented deep learning inference engine image built in step 2 from a public image repository Docker Hub, and enter the container internal environment, run the deep learning model exported in step 3 in the basic running environment of the container, and based on the Docker buildx command, build the container running the deep learning model into a new image in a cross-compiled manner, and the image is a deep learning model image for a specified task, which runs in an embedded device system environment with an ARM processor.

[0061] Based on the content of the above device embodiment, as an optional embodiment, the ARM processor-oriented deep learning model deployment device provided in the embodiment of the application further includes: a fifth sub-module, configured to implement step 5 and specifically configured to: prepare an embedded device with an ARM processor, configure a suitable operating system for the device, install the required basic components of Docker and add a GPG key to it, and according to the specific situation of the operating system, install the current system supported Docker version.

[0062] Based on the content of the above device embodiment, as an optional embodiment, the ARM processor-oriented deep learning model deployment device provided in the embodiment of the application further includes a sixth sub-module configured to implement step 6, which specifically includes: pulling the deep learning model image for the specified task built in step 4 from the public image warehouse Docker Hub on the embedded device for model deployment, entering the container environment for model inference by executing a Docker start command on the image, inserting a neural computing stick into the USB port of the embedded device for accelerating model inference, inputting specified data into the model for inference prediction, and completing the deployment of the ARM processor-oriented deep learning model.

[0063] The method of the embodiment of the application is implemented by relying on an electronic device, and therefore it is necessary to introduce the related electronic device. For this purpose, the embodiment of the application provides an electronic device, as shown in the accompanying drawings, which includes at least one processor, a communications interface, at least one memory, and a communications bus, wherein the at least one processor, the communications interface, and the at least one memory complete mutual communication through the communications bus. The at least one processor can invoke the logical instructions in the at least one memory to execute all or part of the steps of the method provided in each method embodiment. Figure 3

[0064] In addition, the logical instructions in the at least one memory described above can be implemented in the form of a software functional unit and sold or used as an independent product, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the application or the part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each method embodiment of the application. The storage medium described above includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0065] ​The apparatus embodiments described above are only illustrative, and the units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0066] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and a necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0067] The flowcharts and block diagrams in the drawings show the possible implementation architecture, function and operation of the systems, methods and computer program products according to the embodiments of the present application. Based on this understanding, each block in the flowchart or block diagram can represent a module, a program segment or a part of code, which includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders from that shown in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0068] It should be noted that the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed, or inherent to such processes, methods, articles or devices. Without more limitations, the elements defined by the statement "comprising" do not exclude the presence of additional identical elements in the processes, methods, articles or devices including the elements.

[0069] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An ARM processor-oriented deep learning model deployment method, characterized in that, Comprise: Step 1, install Docker Desktop on a local electronic device; step 2, cross-compile a deep learning inference engine for ARM processors; Step 3, train a deep learning model for a specified task and export the trained deep learning model; step 4, build a deep learning model image for a specified task; step 5, install Docker on an embedded device; step 6, pull the previously trained deep learning model in the public image repository Docker Hub, and deploy and infer the trained deep learning model on the embedded device; Step 1 specifically includes: installing Docker Desktop on a local Windows computer, and enabling WSL in the Windows system, running a Linux kernel in a Windows virtual machine through virtualization technology, enabling cross-compilation in the virtual environment of Docker Desktop, and compiling executable image files for ARM processors on AMD processors; Step 2 specifically includes: in the cross-compilation environment enabled in step 1, according to the open source OpenVINO source code and its Dockerfile file, cross-compile a deep learning inference engine image for ARM processors; build a basic runtime environment for a variety of mainstream deep learning frameworks and classic deep learning models in the engine image, provide support for Ubuntu operating system, Python programming language, OpenCV computer vision framework, Pytorch and Tensorflow deep learning framework environment dependencies, and push the engine image to the public image repository Docker Hub, to realize the rapid acquisition of the image when deploying the deep learning model; Step 3 specifically includes: preparing data for deep learning model training, and training a deep learning model on a local Windows computer based on a mainstream deep learning framework, reading the trained model file and re-exporting it as an OpenVINO supported model format, including an ONNX model or an IR model of OpenVINO, the IR model consists of two files with the same name, one is a.bin format file containing network topology information, and the other is an.xml format file containing weight and bias binary data; Step 4 specifically includes: based on the local Docker Desktop software, pull the deep learning inference engine image for ARM processors built in step 2 from the public image repository Docker Hub, and enter its container internal environment, run the deep learning model for a specified task in the basic runtime environment of the container, and based on the Docker buildx command, build a new image by running the container with the deep learning model in a cross-compiled manner, which is a deep learning model image for a specified task, running in the embedded device system environment with ARM processors.

2. The ARM processor-oriented deep learning model deployment method according to claim 1, wherein, Step 5 specifically includes: preparing an embedded device with an ARM processor, and configuring a suitable operating system for the device, installing the required basic components of Docker and adding a GPG key, and installing the current system-supported Docker version according to the specific circumstances of the operating system. 3.The ARM processor-oriented deep learning model deployment method of claim 2, wherein, Step 6 specifically includes: on the embedded device for model deployment, pulling the deep learning model image for the specified task built in step 4 from the public image repository Docker Hub, entering the container environment for model inference by executing the Docker start command on the image, inserting a neural computing stick into the USB port of the embedded device for model inference acceleration, inputting specified data into the model for inference prediction, and completing the deployment of the deep learning model for the ARM processor.

4. An ARM processor-oriented deep learning model deployment apparatus, characterized in that, Comprise: A first main module for implementing step 1, installing Docker Desktop on a local electronic device; step 2, cross-compiling a deep learning inference engine for an ARM processor; A second main module for implementing step 3, training a deep learning model for a specified task and exporting the trained deep learning model; A third main module for implementing step 4, building a deep learning model image for a specified task; step 5, installing Docker on an embedded device; A fourth main module for implementing step 6, pulling the previously trained deep learning model from the public image repository Docker Hub, and deploying and inferring the trained deep learning model on an embedded device; The first main module is specifically configured to install Docker Desktop on a local Windows computer, and to enable WSL in the Windows system, to run a Linux kernel in a Windows virtual machine through virtualization technology, and to enable cross-compilation in the virtual environment of DockerDesktop to compile an executable image file for an ARM processor on an AMD processor; Under the cross-compilation environment enabled in step 1, the deep learning inference engine image for the ARM processor is built in a cross-compiled manner based on the open-source OpenVINO source code and its Dockerfile file; In the engine image, a basic running environment for various mainstream deep learning frameworks and classic deep learning models is built, providing support for Ubuntu operating system, Python programming language, OpenCV computer vision framework, Pytorch and Tensorflow deep learning framework environment dependencies, and the engine image is pushed to the public image repository Docker Hub to enable fast access to the image during deep learning model deployment; The second main module is specifically used for preparing data for deep learning model training, training a deep learning model on a GPU of a local Windows computer based on a mainstream deep learning framework, reading a trained model file and re-exporting the model file into a model format supported by OpenVINO, including an ONNX model or an IR model of OpenVINO, and the IR model is composed of two files with the same name, one is a.bin format file containing network topology information, and the other is an.xml format file containing weight and bias binary data; The third main module is specifically used for pulling, based on a local Docker Desktop software, a deep learning inference engine image for an ARM processor built in step 2 from a public image warehouse DockerHub, entering an internal environment of the container, running the deep learning model for a specified task exported in step 3 in a basic running environment of the container, and building the container running the deep learning model into a new image in a cross-compiled manner based on a Docker buildx command, and the image is a deep learning model image for the specified task, which runs in an embedded device system environment with an ARM processor.

5. An electronic device, comprising: Comprise: at least one processor, at least one memory and a communication interface; wherein, the processor, memory and communication interface communicate with each other; the memory stores program instructions executable by the processor, and the processor invokes the program instructions to execute the method of any one of claims 1 to 3.

6. A non-transitory computer-readable storage medium, comprising, The non-transitory computer readable storage medium stores computer instructions, and the computer instructions make the computer execute the method of any one of claims 1 to 3.

Images