Neural network calculation acceleration method based on edge end heterogeneous distributed device

Abstract

The invention discloses a neural network calculation acceleration method based on edge end heterogeneous distributed equipment. The method comprises the following steps: initializing an environment; the master device divides the neural network to-be-calculated data and sends the data to the slave device; each device in the edge end heterogeneous distributed device group controls a self accelerator to perform neural network acceleration calculation, the edge end heterogeneous distributed device group comprises a master device and a slave device, and the accelerator comprises an FPGA, an NPU and / or a DSP; and the slave device sends the acceleration calculation result to the master device. According to the method, the heterogeneous distributed device group can be formed by a plurality of devices with different architectures such as the FPGA, the NPU and the DSP, neural network calculation is accelerated cooperatively, the neural network calculation is accelerated by the edge end heterogeneous distributed device, and the devices in the group are flexibly coordinated, so that the compatibility and flexibility of the edge end neural network acceleration system are improved, and the calculation speed of the neural network is improved. And low delay and high performance are ensured.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more specifically to a neural network computing acceleration method based on edge heterogeneous distributed devices. Background Art

[0002] In recent years, artificial intelligence technology based on neural networks has developed rapidly. Due to its good performance, artificial intelligence has been widely used in target detection, medical information processing, natural language processing and other fields. With the vigorous development of the artificial intelligence industry, the demand for applying artificial intelligence technology on devices in resource-constrained scenarios is gradually increasing, such as security monitoring and intelligent driving. Although neural networks perform well in many artificial intelligence applications, they come at the cost of high computational complexity.

[0003] Neural network computing is mainly matrix computing. Traditionally, cloud computing is usually used to accelerate neural network computing. However, cloud computing has high transmission latency and is difficult to support many real-time artificial intelligence applications with high computational complexity. Edge computing is to analyze and process the data collected from the terminal directly in the local device or network close to the data generation, without transmitting the data to the cloud data processing center. Edge neural network computing acceleration devices usually use FPGA (Field-Programmable Gate Array), NPU (Neural Processing Unit), DSP (Digital Signal Processor), etc. FPGA is a programmable logic gate array that accelerates neural network computing by utilizing the parallel processing capability of FPGA. NPU is a processor designed specifically for accelerating neural network computing tasks. A special neural network computing module is designed according to the neural network model and optimized for various computing requirements in the neural network. DSP is a microprocessor specifically used for digital signal processing. It has a special hardware multiplier that can quickly perform multiplication operations in neural networks. These three architectures have the characteristics of high energy efficiency for neural network computing. For example, the patent "A neural network acceleration method based on collaborative processing of multiple FPGAs" uses multiple FPGAs to collaboratively process and accelerate neural networks. The patent "Target detection method, NPU, AI edge computing device and storage medium" deploys NPU at the edge to complete the computational acceleration of target detection neural networks. The patent "A neural network reasoning task division method based on multi-core DSP hardware characteristics" uses multi-core DSPs for collaborative reasoning of neural networks.

[0004] However, the prior art has the following disadvantages:

[0005] (1) Single-device acceleration at the edge: A single device at the edge usually uses a single FPGA, NPU, or DSP. Accelerating the neural network on the FPGA or using the NPU or DSP has the advantages of low power consumption and high energy efficiency. The entire system structure design is relatively simple. In addition, since the calculation acceleration process is performed within a single device, the neural network calculation is not affected by the data transmission delay between devices. However, the absolute performance of a single device is relatively weak.

[0006] (2) Edge homogeneous distributed device acceleration: Edge distributed device acceleration usually uses distributed FPGA, NPU or DSP. Its acceleration method is to map computing tasks to each device node based on a single device at the edge. By designing an efficient data communication mechanism, it ensures high-speed data transmission between each device node and between the device checkpoint and the CPU of the end device. Compared with cloud computing, it can reduce the delay of data transmission back and forth, and has stronger performance than a single device at the edge. It can solve more real-time artificial intelligence applications with high computational complexity. However, the edge usually uses homogeneous distributed devices, that is, the accelerator in the system is only FPGA, NPU or DSP, and these devices are not coordinated for computing acceleration, so the compatibility and flexibility of the whole system are low. Summary of the invention

[0007] In view of this, the present invention provides a neural network computing acceleration method based on edge heterogeneous distributed devices, which can combine multiple FPGAs, NPUs, DSPs and other devices with different architectures into a heterogeneous distributed device group to collaboratively accelerate neural network computing, thereby realizing the acceleration of neural network computing by edge heterogeneous distributed devices and flexibly coordinating devices within the group, thereby improving the compatibility and flexibility of the edge neural network acceleration system and ensuring low latency and high performance.

[0008] In order to achieve the above object, the present invention adopts the following technical solution:

[0009] The neural network computing acceleration method based on edge heterogeneous distributed devices includes:

[0010] Environment initialization;

[0011] The master device divides the data to be calculated by the neural network and sends it to the slave device;

[0012] Each device in the edge heterogeneous distributed device group controls its own accelerator to perform neural network acceleration calculations. The edge heterogeneous distributed device group includes a master device and a slave device. The accelerator includes an FPGA, an NPU and / or a DSP.

[0013] The slave device sends the accelerated calculation results to the master device.

[0014] Preferably, the environment initialization specifically includes:

[0015] The edge device that generates the neural network computing demand is set as the master device, and other edge heterogeneous distributed devices are set as slave devices;

[0016] The processor of the master device queries the slave device information through the communication module, and the master device and the slave device update the corresponding message in the registration table, and the registration table is stored in the memory of each device;

[0017] The processor of the master device initializes the distributed environment by running the script. The processor of the master device sends instructions to the processors of each slave device, loads the neural network model script in the slave device memory, activates the accelerator on each slave device, and sends test instructions to the processors of each slave device to detect the communication connection between the master device and each slave device in the heterogeneous distributed device group at the edge to ensure the stability of the connection between the master and slave devices in the heterogeneous distributed device group at the edge.

[0018] Preferably, the master device divides the neural network data to be calculated and sends it to the slave device, specifically including:

[0019] The processor of the master device divides the neural network data to be calculated into N parts according to the number of master devices and slave devices and the computing performance of the accelerators of each device, where N is the total number of master and slave devices;

[0020] The processor of the master device controls the communication module to send N-1 copies of data to the corresponding slave devices in the edge heterogeneous distributed device group in sequence.

[0021] Preferably, for a device whose accelerator architecture is FPGA, a neural network calculation acceleration circuit has been burned on the FPGA chip, and the neural network weight parameters are stored in the memory. The processor controls the operation of each circuit on the FPGA through the neural network model script, and the neural network data to be calculated and the neural network weight parameters are loaded onto the FPGA through the accelerator cache for accelerated calculation, and the calculation results are stored in the memory through the cache.

[0022] For devices whose accelerator architecture is NPU, the processor accelerates the calculation of the matrix by the neural network calculation module of NPU through the neural network model script. The neural network data to be calculated and the neural network weight parameters are loaded to NPU through the accelerator cache for accelerated calculation, and the calculation results are saved in the memory through the cache.

[0023] For devices whose accelerator architecture is DSP, the processor controls the DSP's hardware multiplier through the neural network model script to accelerate the matrix multiplication calculation. The neural network data to be calculated and the neural network weight parameters are loaded onto the DSP through the accelerator cache for accelerated calculation, and the calculation results are saved in the memory through the cache.

[0024] Preferably, the processor of the master device queries the slave device information through the communication module, including the address of each slave device, the total number of slave devices, the accelerator status of each slave device, and the accelerator performance of each slave device.

[0025] Preferably, it also includes:

[0026] When the master device queries the slave device information and finds a new device to be expanded, the new device information is added to the master device registry and synchronized to the registry of other slave devices in the edge heterogeneous distributed device group.

[0027] Preferably, it also includes:

[0028] When the master device queries the slave device information and finds that the original slave device fails or exits the system, the master device deregisters the information of this device in the registry and synchronizes it to other slave devices in the heterogeneous distributed device group at the edge, interrupting the communication connection between this device and the master device.

[0029] Preferably, the master device and the slave device of the edge heterogeneous distributed device group have the same structure, both including a processor, an accelerator, a memory, a communication module and a bus.

[0030] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a neural network computing acceleration method based on edge heterogeneous distributed devices, which has the following advantages:

[0031] (1) Compatibility: The present invention is not limited to the use of homogeneous distributed device groups to accelerate neural network calculations. It can be effectively compatible with devices of different architectures. It can accelerate neural networks by combining FPGA, NPU and DSP devices of different architectures into heterogeneous distributed device groups, thereby reducing the system's dependence on computing devices of specific architectures, facilitating the expansion of system computing resources, and improving the compatibility of the neural network calculation acceleration system.

[0032] (2) Flexibility: The design method of the present invention allows the expansion of the distributed edge device group, as well as the flexible adjustment of the allocation of computing resources and the flexible control of data transmission according to actual application needs, which can meet performance and latency requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0034] Figure 1 A flow chart of the neural network computing acceleration method based on edge heterogeneous distributed devices provided by the present invention.

[0035] Figure 2 This is a communication structure diagram of the master device and slave device provided by the present invention.

[0036] Figure 3 This is a structural diagram of the edge device provided by the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] The embodiment of the present invention discloses a neural network computing acceleration method based on edge heterogeneous distributed devices, such as Figure 1 As shown, including:

[0039] Environment initialization;

[0040] The master device divides the data to be calculated by the neural network and sends it to the slave device;

[0041] Each device in the edge heterogeneous distributed device group controls its own accelerator to perform neural network acceleration calculations. The edge heterogeneous distributed device group includes master devices and slave devices, such as Figure 2 As shown, the accelerator includes an FPGA, an NPU and / or a DSP;

[0042] The slave device sends the accelerated calculation results to the master device.

[0043] Among them, the master device and the slave device of the edge heterogeneous distributed device group have the same structure, including processors, accelerators, memories, communication modules and buses, such as Figure 3 shown.

[0044] 1. Processor

[0045] (1) Processor core: responsible for executing instructions and processing data.

[0046] During the environment initialization phase, the master device processor queries the information of other slave devices, and the master and slave devices update the registry; initialize the distributed environment by running the script; load the neural network model script stored in the memory of the slave device by sending instructions to each slave device processor, and activate the accelerator on the slave device; send test instructions to detect the stability of the connection between the master and slave node devices in the distributed device group.

[0047] In the data division and transmission stage, the master device processor divides the neural network data to be calculated and transmits it to the slave device.

[0048] During the neural network calculation stage, each device in the edge heterogeneous distributed device group controls the accelerator to accelerate the neural network calculation.

[0049] In the result aggregation stage, the slave device processor controls the communication module to aggregate the output results calculated by each slave device to the master device.

[0050] (2) Cache: used to store temporary data to increase data access speed.

[0051] (3) Bus interface: responsible for communication and data exchange between the processor and other modules.

[0052] 2. Accelerator

[0053] (1) Accelerator core: There are three different architectures: FPGA, NPU and DSP. In the neural network calculation stage, the processor controls the completion of neural network data acceleration calculation.

[0054] (2) Cache: used to store temporary data to increase data access speed.

[0055] (3) Bus interface: responsible for communication and data exchange between the accelerator and other modules.

[0056] 3. Memory

[0057] It is used to store neural network model scripts, neural network data to be calculated, neural network weight parameters and calculation results suitable for the local device accelerator architecture. It contains a bus interface for communication and data exchange between the memory and other modules.

[0058] 4. Communication module

[0059] Used for communication and data exchange between devices in a heterogeneous distributed device group at the edge, connected to other devices in the device through a bus for communication and data exchange.

[0060] 5. Bus

[0061] High-speed bus. The processor, accelerator, memory and communication module on the edge device are connected through the bus to transmit instructions and data.

[0062] The specific process of the present invention is described in detail below:

[0063] 1) Environment initialization

[0064] The edge device that generates the neural network computing demand is set as the master node device, referred to as the master device, and the other edge devices are set as slave node devices, referred to as slave devices. The master device processor queries the slave device information through the communication module, including the address of each slave device, the total number of slave devices, the status of each slave device accelerator, and the performance of each slave device accelerator. The master device and the slave device update the corresponding message in the registry, and the registry information is stored in the memory of each device. After that, the master device processor initializes the distributed environment by running the script. The master device processor sends instructions to each slave device processor, loads the neural network model script stored in the memory of the slave device, and activates the accelerator on each slave device. The master device processor sends test instructions to each slave device processor to detect the communication connection between the master and slave devices in the distributed device group to ensure the stability of the master and slave device connection in the distributed device group. Figure 2 The communication connection structure diagram of the master device and the slave device. The master and slave devices are connected through the communication module in a wired or wireless manner.

[0065] When the master device queries the slave device information, it finds a new device to be expanded, adds the new device information to the master device registry, and synchronizes it to the registry of other slave devices in the distributed edge device group. The new edge device must have the same structure as other edge devices, that is, it has a processor, accelerator, memory, communication module, and bus design. If the original slave device fails or exits the system, the master device cancels the information of this device in the registry and synchronizes it to other slave devices, interrupting the communication connection between this device and the master device. The above method can improve the flexibility of the system.

[0066] 2) Data division and transmission

[0067] The master device processor divides the neural network data to be calculated into N parts according to the number of master devices and slave devices and the computing performance of the accelerator. Then the master device processor controls the communication module to send N-1 parts of the data to the corresponding slave devices in the edge heterogeneous distributed device group in sequence.

[0068] 3) Neural network calculation

[0069] The memory of each device in the edge heterogeneous distributed device group stores the neural network model script suitable for the accelerator architecture of this device. After the calculation data is imported into each device, the processor on each device controls the accelerator to accelerate the neural network calculation.

[0070] For devices with FPGA as the accelerator architecture, the neural network computing acceleration circuit has been burned into the FPGA chip, and the neural network weight parameters are stored in the memory. The processor controls the operation of various circuits on the FPGA through the neural network model script. The neural network data to be calculated and the neural network weight parameters are loaded onto the FPGA through the accelerator cache for accelerated calculation, and the results are saved in the memory through the cache.

[0071] For devices with an accelerator architecture of NPU, the processor accelerates the calculation of the matrix by the neural network computing module of the NPU through the neural network model script. The neural network data to be calculated and the neural network weight parameters are loaded to the NPU through the accelerator cache for accelerated calculation, and the calculation results are saved in the memory through the cache.

[0072] For devices whose accelerator architecture is DSP, the processor controls the DSP's hardware multiplier through the neural network model script to accelerate the matrix multiplication calculation. The neural network data to be calculated and the neural network weight parameters are loaded onto the DSP through the accelerator cache for accelerated calculation, and the calculation results are saved in the memory through the cache.

[0073] 4) Results summary

[0074] After each slave device completes calculation, the slave device processor controls the communication device to summarize the output results of the calculations of each slave device to the master device. To avoid bus congestion caused by simultaneous transmission to the master device, each slave device transmits part of the results back to the master device after processing part of the data.

[0075] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0076] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A neural network computing acceleration method based on edge heterogeneous distributed devices, characterized in that: include: Environment initialization; The master device divides the data to be calculated by the neural network and sends it to the slave device; Each device in the edge heterogeneous distributed device group controls its own accelerator to perform neural network acceleration calculations. The edge heterogeneous distributed device group includes a master device and a slave device. The accelerator includes an FPGA, an NPU and / or a DSP. The slave device sends the accelerated calculation results to the master device.

2. The neural network computing acceleration method based on edge heterogeneous distributed devices according to claim 1 is characterized in that: Environment initialization, including: The edge device that generates the neural network computing demand is set as the master device, and other edge heterogeneous distributed devices are set as slave devices; The processor of the master device queries the slave device information through the communication module, and the master device and the slave device update the corresponding message in the registration table, and the registration table is stored in the memory of each device; The processor of the master device initializes the distributed environment by running the script. The processor of the master device sends instructions to the processors of each slave device, loads the neural network model script in the slave device memory, activates the accelerator on each slave device, and sends test instructions to the processors of each slave device to detect the communication connection between the master device and each slave device in the heterogeneous distributed device group at the edge to ensure the stability of the connection between the master and slave devices in the heterogeneous distributed device group at the edge.

3. The neural network computing acceleration method based on edge heterogeneous distributed devices according to claim 1 is characterized in that: The master device divides the neural network data to be calculated and sends it to the slave device, including: The processor of the master device divides the neural network data to be calculated into N parts according to the number of master devices and slave devices and the computing performance of the accelerators of each device, where N is the total number of master and slave devices; The processor of the master device controls the communication module to send N-1 copies of data to the corresponding slave devices in the edge heterogeneous distributed device group in sequence.

4. The neural network computing acceleration method based on edge heterogeneous distributed devices according to claim 1 is characterized in that: For devices with FPGA as the accelerator architecture, the neural network calculation acceleration circuit has been burned on the FPGA chip, and the neural network weight parameters are stored in the memory. The processor controls the operation of each circuit on the FPGA through the neural network model script. The neural network data to be calculated and the neural network weight parameters are loaded onto the FPGA through the accelerator cache for accelerated calculation, and the calculation results are saved in the memory through the cache. For devices whose accelerator architecture is NPU, the processor uses the neural network model script to accelerate the calculation of the matrix by the neural network calculation module of NPU. The neural network data to be calculated and the neural network weight parameters are loaded to NPU through the accelerator cache for accelerated calculation. The calculation results are saved in the memory through the cache. For devices whose accelerator architecture is DSP, the processor controls the DSP's hardware multiplier through the neural network model script to accelerate the matrix multiplication calculation. The neural network data to be calculated and the neural network weight parameters are loaded onto the DSP through the accelerator cache for accelerated calculation, and the calculation results are saved in the memory through the cache.

5. The neural network computing acceleration method based on edge heterogeneous distributed devices according to claim 2 is characterized in that: The processor of the master device queries the slave device information through the communication module, including the address of each slave device, the total number of slave devices, the accelerator status of each slave device, and the accelerator performance of each slave device.

6. The neural network computing acceleration method based on edge heterogeneous distributed devices according to claim 2 is characterized in that: Also includes: When the master device queries the slave device information and finds a new device to be expanded, the new device information is added to the master device registry and synchronized to the registry of other slave devices in the heterogeneous distributed device group at the edge.

7. The neural network computing acceleration method based on edge heterogeneous distributed devices according to claim 2 or 6 is characterized in that: Also includes: When the master device queries the slave device information and finds that the original slave device fails or exits the system, the master device deregisters the information of this device in the registry and synchronizes it to other slave devices in the heterogeneous distributed device group at the edge, interrupting the communication connection between this device and the master device.

8. The neural network computing acceleration method based on edge heterogeneous distributed devices according to claim 1 is characterized in that: The master device and slave devices of the heterogeneous distributed device group at the edge have the same structure, including processors, accelerators, memories, communication modules and buses.

Images