Method and system for cross-system calling of artificial intelligence computing power

Abstract

The invention discloses a cross-system artificial intelligence computing power calling method and system, and belongs to the technical field of artificial intelligence. The method comprises the following steps: S1, creating a shared memory; s2, starting a drive container and a virtual NPU rear-end drive; step S3, starting a virtual NPU front-end drive; and step S4, running the OpenHarmony native application. The method for calling the artificial intelligence computing power is provided for the OpenHarmony native application, and a computing power base is provided for the OpenHarmony intelligent application. According to the scheme of the invention, the OpenHarmony native application does not need to be transformed and intrusively modified, so that the method has great expansibility and can be adapted to various different artificial intelligence computing power hardware; therefore, the intelligent development of the terminal can be quickly promoted.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a method and system for cross-system calling artificial intelligence computing power. Background Art

[0002] With the emergence of large models, intelligent applications have an increasingly strong demand for AI computing power, and traditional general-purpose computing power can no longer meet such demands. Unlike traditional general-purpose computing, AI computing requires the use of dedicated computing hardware such as GPUs and NPUs to provide AI computing power. Accordingly, the operating system needs to provide drivers for accessing hardware such as GPUs and NPUs, as well as corresponding APIs for application calls.

[0003] Currently, mainstream GPU and NPU device manufacturers that provide powerful AI computing power mainly provide drivers and APIs for large computers such as servers, which usually run Linux or Windows operating systems. In the embedded field, especially in the field of mobile terminal devices, Android or OpenHarmony operating systems are usually run; however, these operating systems lack drivers for AI computing hardware and support for corresponding APIs, which greatly limits the intelligent application innovation of terminal devices. This requires enabling the OpenHarmony operating system to call mainstream GPU and NPU devices. Summary of the invention

[0004] To achieve the above object, the present invention is a method for cross-system calling artificial intelligence computing power based on shared memory, and the method comprises the following steps: Step S1, creating a shared memory; Step S2, start the driver container and the virtual NPU backend driver; Step S3, starting the virtual NPU front-end driver; Step S4, running the OpenHarmony native application.

[0005] As an improvement of the present invention, in step S1, the OpenHarmony system creates a shared memory through mmap, and initializes a control channel and a data channel in the shared memory.

[0006] As an improvement of the present invention, in step S2, the OpenHarmony system starts the driver container. During the process of starting the driver container, the structural information of the shared memory needs to be passed to the driver container. At this time, the virtual NPU back-end driver will obtain the information of the control channel and the data channel for later communication with the virtual NPU front-end driver.

[0007] As an improvement of the present invention, in step S3, after completing the startup of the driver container, OpenHarmony will start the virtual NPU front-end driver. At this time, it is necessary to pass the structural information of the shared memory to the NPU front-end driver. The structural information contains the shared memory address, as well as the description information of the control channel and the data channel, which is used for later communication with the virtual NPU back-end driver.

[0008] As an improvement of the present invention, in step S4, since the virtual NPU front-end driver provides an API interface compatible with the OpenHarmony application framework, the OpenHarmony native application calls the NPU computing power by calling the API of the OpenHarmony application framework. At this time, the virtual NPU front-end driver will convert the API call into a call of the virtual NPU back-end driver through the shared memory mechanism; and the back-end driver realizes the call to the physical NPU by directly calling the Linux NPU driver.

[0009] As an improvement of the present invention, according to the cross-system call artificial intelligence computing power system based on shared memory used by the method, the system includes a driver container, in which the Linux NPU driver is placed to realize access control to the NPU hardware; a virtual NPU driver, which is divided into two parts, namely the virtual NPU front-end driver and the virtual NPU back-end driver, which communicate through shared memory; shared memory, which is used to realize data intercommunication between the driver container and the NPU virtual driver on the OpenHarmony side.

[0010] As an improvement of the present invention, the virtual NPU front-end driver is connected to the application framework of OpenHarmony to provide a driver implementation compatible with the OpenHarmony driver framework; the driver is connected to the virtual NPU back-end driver through shared memory; and the NPU hardware device is indirectly accessed through the virtual NPU back-end driver.

[0011] As an improvement of the present invention, the virtual NPU backend driver is connected to the Linux NPU driver and connected to the virtual NPU front-end driver through shared memory; and the access request of the virtual NPU front-end driver is converted into a call to the Linux NPU driver, thereby realizing access to the NPU hardware.

[0012] As an improvement of the present invention, the shared memory includes a control channel, which occupies a fixed area in the shared memory and is used to realize the command transmission of the front-end and back-end drivers of the virtual NPU. The command transmission is realized by RPC mode, and the RPC mechanism implementation steps include: When the front-end driver needs to call and send a control command to the back-end driver, the front-end driver writes the command type and data into the shared memory area occupied by the control channel and sends a signal; After starting, the back-end driver is in a state of waiting for a signal. When it gets a signal from the front-end driver, it reads the command information from the shared memory area occupied by the control channel. After the back-end driver reads the command information, it calls the function of the Linux NPU driver according to the command information, and returns the call result to the front-end driver through the control channel.

[0013] As an improvement of the present invention, the data channel includes a data channel. Since artificial intelligence calculation involves a large amount of data transmission, it is necessary to transfer these data between the front-end and back-end drivers and the Linux NPU driver; the dynamic shared memory application process includes: When the application needs to submit AI data to the NPU for use, the front-end driver allocates a section of memory in the shared memory area according to the demand; The front-end driver writes the data to the shared memory area and informs the back-end driver of the address of the memory area through the control channel; After the backend driver learns the address of the shared memory area, it sends the memory area and data to the NPU hardware device through the standard interface of the Linux NPU driver.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention provides a method for calling artificial intelligence computing power for OpenHarmony native applications, and provides a computing power base for OpenHarmony intelligent applications. The present invention does not require transformation and intrusive modification of OpenHarmony native applications, has great scalability, and can adapt to various different artificial intelligence computing power hardware; thereby, it can quickly promote the development of terminal intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a comparison chart between Linux and OpenHarmony; Figure 2 This is the architecture diagram of the artificial intelligence computing system of the present invention; Figure 3 This is a schematic diagram of the control channel process of the present invention; Figure 4 This is a schematic diagram of the data channel process of the present invention; Figure 5 The present invention completes the OpenHarmony native application calling NPU computing power flow chart. DETAILED DESCRIPTION

[0016] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0017] Embodiment: As Ascend 310 NPU does not provide OpenHarmony-oriented drivers and corresponding APIs, applications on OpenHarmony cannot directly call these drivers; in this solution, this problem is solved through a shared memory solution. Figure 1 By comparing Linux and OpenHarmony, the current problems are pointed out.

[0018] Ascend 310 NPU provides drivers and APIs for Linux operating systems, and OpenHarmony uses the same kernel technology as the Linux operating system, which provides a technical basis for solving this problem. In this solution, OpenHarmony uses containers and shared memory to implement the function of calling Ascend 310 NPU artificial intelligence computing power. Its architecture is as follows: Figure 2 shown.

[0019] According to the architecture diagram of this embodiment, related interfaces are defined according to modules.

[0020] The shared memory structure is described as follows:

[0021] Shared memory should provide the following interface.

[0022]

[0023] The driver container should provide the following interface.

[0024]

[0025] The key components and functions of the cross-system call artificial intelligence computing system based on shared memory described in this embodiment include: driver container. Since the driver framework of OpenHarmony is different from that of Linux, the Linux NPU driver cannot be used directly in OpenHarmony. In this solution, a driver container is designed on the OpenHarmony system, and the Linux NPU driver is placed in the container to implement access control to the NPU hardware.

[0026] Virtual NPU driver, OpenHarmony application, is developed using the API provided by OpenHarmony's application framework. These APIs are compatible with OpenHarmony's driver framework, but not with Linux's driver framework, so Linux's NPU driver cannot be called directly. In order to ensure native support for OpenHarmony applications, a virtual NPU driver is designed in this solution. The driver is divided into two parts, namely the virtual NPU front-end driver and the virtual NPU back-end driver. They communicate through shared memory, and their functions are as follows: Virtual NPU front-end driver: referred to as front-end driver, the front-end driver is connected to the OpenHarmony application framework to provide a driver implementation compatible with the OpenHarmony driver framework; the driver does not directly access the NPU hardware device, but is connected to the virtual NPU back-end driver through shared memory; the NPU hardware device is indirectly accessed through the virtual NPU back-end driver.

[0027] Virtual NPC back-end driver: referred to as back-end driver, the back-end driver is connected to the Linux NPU driver and connected to the virtual NPU front-end driver through shared memory; and the access request of the virtual NPU front-end driver is converted into a call to the Linux NPU driver, thereby realizing access to the NPU hardware.

[0028] Shared memory: Shared memory is used to achieve data communication between the driver container and the NPU virtual driver on the OpenHarmony side. Since the driver container and the OpenHarmony NPU virtual driver run on the same Linux kernel, shared memory can be created through mmap. Shared memory is divided into two parts, namely the control channel and the data channel. Its functions include: Combination Figure 3 ,Control channel: The control channel occupies a fixed area in the shared memory and is used to implement command transmission between the front-end and back-end drivers of the virtual NPU. The transmission of commands is implemented through RPC. The implementation process of the RPC mechanism is as follows: When the front-end driver needs to call and send a control command to the back-end driver, the front-end driver writes the command type and data into the shared memory area occupied by the control channel and sends a signal; After starting, the back-end driver is in a state of waiting for signals. When it obtains the signal from the front-end driver, it reads the command information, including type, data, etc., from the shared memory area occupied by the control channel. After the back-end driver reads the command information, it calls the function of the Linux NPU driver according to the command information, and returns the call result to the front-end driver through the control channel.

[0029] Combination Figure 4 ,Data channel: Since artificial intelligence calculations involve a large amount of data transmission, it is necessary to transfer this data between the front-end and back-end drivers and the Linux NPU driver; in order to reduce the time overhead in the data transmission process, in this solution, shared memory is used for data transmission. Since the size of this part of the memory is uncertain, in this solution, dynamic shared memory application is supported, and the process is as follows: when the application needs to submit artificial intelligence data to the NPU for use, the front-end driver will open up a section of memory in the shared memory area according to the demand; the front-end driver writes the data to the shared memory area, and informs the back-end driver of the address of the memory area through the control channel; after the back-end driver learns the address of the shared memory area, it sends the memory area and data to the NPU hardware device through the standard interface of the Linux NPU driver.

[0030] The process of OpenHarmony native application calling NPU computing power is as follows: Creating shared memory The OpenHarmony system creates shared memory through mmap and initializes the control channel and data channel in the shared memory.

[0031] Start the driver container and virtual NPU backend driver The OpenHarmony system starts the driver container, which contains the Linux NPU driver and the virtual NPU backend driver. In the process of starting the driver container, the structure information of the shared memory needs to be passed to the driver container. The structure information contains the shared memory address and the description information of the control channel and data channel, such as location, size, etc. At this time, the virtual NPU backend driver will obtain the information of the control channel and data channel, which can be used for communication with the virtual NPU front-end driver later.

[0032] Start the virtual NPU front-end driver After completing the startup of the driver container, OpenHarmony will start the virtual NPU front-end driver. At this time, the structure information of the shared memory needs to be passed to the NPU front-end driver. The structure information contains the shared memory address, as well as the description information of the control channel and data channel, such as location, size, etc.; it is used for later communication with the virtual NPU back-end driver.

[0033] Run OpenHarmony native applications Since the virtual NPU front-end driver provides an API interface compatible with the OpenHarmony application framework, the OpenHarmony native application can call the NPU computing power by calling the API of the OpenHarmony application framework. At this time, the virtual NPU front-end driver will convert the API call into a call to the virtual NPU back-end driver through the shared memory mechanism; and the back-end driver will directly call the Linux NPU driver to realize the call to the physical NPU.

[0034] After the shared memory and driver containers are created, follow Figure 5 The process of OpenHarmony native application calling NPU computing power is completed through the process.

[0035] It should be noted that the above content only illustrates the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.

Claims

1. A method for calling artificial intelligence computing power across systems, characterized in that: The method The following steps are included: Step S1, creating a shared memory; Step S2, start the driver container and the virtual NPU backend driver; Step S3, starting the virtual NPU front-end driver; Step S4, running the OpenHarmony native application.

2. The method for calling artificial intelligence computing power across systems according to claim 1, characterized in that: In the step S1, the OpenHarmony system creates a shared memory through mmap, and initializes a control channel and a data channel in the shared memory.

3. The method for calling artificial intelligence computing power across systems according to claim 1, characterized in that: In step S2, the OpenHarmony system starts the driver container. During the process of starting the driver container, the structure information of the shared memory is passed to the driver container. At this time, the virtual NPU back-end driver will obtain the information of the control channel and the data channel for later communication with the virtual NPU front-end driver.

4. The method for calling artificial intelligence computing power across systems according to claim 1, characterized in that: In step S3, after completing the startup of the driver container, OpenHarmony will start the virtual NPU front-end driver. At this time, it is necessary to pass the structure information of the shared memory to the NPU front-end driver. The structure information contains the shared memory address, as well as the description information of the control channel and the data channel, which will be used for later communication with the virtual NPU back-end driver.

5. The method for calling artificial intelligence computing power across systems according to claim 1, characterized in that: In step S4, since the virtual NPU front-end driver provides an API interface compatible with the OpenHarmony application framework, the OpenHarmony native application calls the NPU computing power by calling the API of the OpenHarmony application framework. At this time, the virtual NPU front-end driver will convert the API call into a call to the virtual NPU back-end driver through the shared memory mechanism; and the back-end driver will call the physical NPU by directly calling the Linux NPU driver.

6. The cross-system call artificial intelligence computing system used by the method according to any one of claims 1 to 5, characterized in that: The system includes a driver container, in which the Linux NPU driver is placed to implement access control to the NPU hardware; a virtual NPU driver, which is divided into two parts, namely a virtual NPU front-end driver and a virtual NPU back-end driver, which communicate through shared memory; and shared memory, which is used to implement data intercommunication between the driver container and the NPU virtual driver on the OpenHarmony side.

7. The cross-system calling artificial intelligence computing power system according to claim 6, characterized in that: The virtual NPU front-end driver is connected to the OpenHarmony application framework to provide a driver implementation compatible with the OpenHarmony driver framework; the driver is connected to the virtual NPU back-end driver through shared memory; and the NPU hardware device is indirectly accessed through the virtual NPU back-end driver.

8. The cross-system calling artificial intelligence computing power system according to claim 6, characterized in that: The virtual NPU backend driver is connected to the Linux NPU driver and connected to the virtual NPU frontend driver through shared memory; and the access request of the virtual NPU frontend driver is converted into a call to the Linux NPU driver, thereby realizing access to the NPU hardware.

9. The cross-system calling artificial intelligence computing power system according to claim 8, characterized in that: The shared memory includes a control channel, which occupies a fixed area in the shared memory and is used to realize the command transmission of the front-end and back-end drivers of the virtual NPU. The command transmission is realized by RPC mode. The RPC mechanism implementation steps include: When the front-end driver needs to call and send a control command to the back-end driver, the front-end driver writes the command type and data into the shared memory area occupied by the control channel and sends a signal; After starting, the back-end driver is in a state of waiting for a signal. When it gets a signal from the front-end driver, it reads the command information from the shared memory area occupied by the control channel. After the backend driver reads the command information, it calls the function of the Linux NPU driver according to the command information; And return the call result to the front-end driver through the control channel.

10. The cross-system calling artificial intelligence computing power system according to claim 8, characterized in that: The data channel includes a data channel. Since artificial intelligence calculations involve a large amount of data transmission, it is necessary to transfer these data between the front-end and back-end drivers and the LinuxNPU driver; the dynamic shared memory application process includes: When the application needs to submit AI data to the NPU for use, the front-end driver allocates a section of memory in the shared memory area according to the demand; The front-end driver writes the data to the shared memory area and informs the back-end driver of the address of the memory area through the control channel; After the backend driver learns the address of the shared memory area, it sends the memory area and data to the NPU hardware device through the standard interface of the Linux NPU driver.

Images